Welcome!

Welcome to Kirsle.net! This is the personal website of Noah Petherbridge, and it's where my various software projects and web blog lives.

[ RSS Feed | Older > ]
Markdown in Rophako


kirsle
Posted by Noah Petherbridge on Friday, April 18 2014 @ 10:05:02 PM

I've made an update to my Rophako CMS that powers Kirsle.net: it now supports Markdown!

Blog posts can be either written in Markdown, or direct HTML (the old way). In Markdown mode, HTML is currently escaped from the post. I may change this and make it only escape HTML in comments if I find it to be a PITA in the future. ;) Oh, and emoticons still work (they're rendered after Markdown is done).

Markdown works in comments, too, but with some limitations, like I don't allow embedding images. Also, you can write entire pages on your site in Markdown. Just create a page named with a .md file extension that contains Markdown code and it "just works" (there's a built-in markdown.inc.html file in the default site that wraps your rendered document in your web design's layout).

Best of all, I'm using a handful of extensions to the Python Markdown module to enable a lot of Github style Markdown features, such as code highlighting. Here's a Python example:

def render_markdown(body, html_escape=True):
    """Render a block of Markdown text.

    This will default to escaping literal HTML characters. Set
    `html_escape=False` to trust HTML."""

    args = dict(
        lazy_ol=False, # If a numbered list starts at e.g. 4, show the <ol> there
        extensions=[
            "fenced_code",  # GitHub style code blocks
            "tables",       # http://michelf.ca/projects/php-markdown/extra/#table
            "smart_strong", # Handles double__underscore better.
            "codehilite",   # Code highlighting with Pygment!
            "nl2br",        # Line breaks inside a paragraph become <br>
            "sane_lists",   # Make lists less surprising
        ],
        extension_configs={
            "codehilite": {
                "linenums": False,
            }
        }
    )
    if html_escape:
        args["safe_mode"] = "escape"

    return markdown.markdown(body, **args)

I'll be revisiting all my old blog posts that have code pasted in them and reformatting them in Markdown for easier maintenance. My old way of pasting code in a blog post was to open the code in vim, and then run commands like,

:let html_use_css=1
:TOhtml

... which would output an HTML file that syntax highlights the code, and then I'd cut and paste all that junk into my blog. Blech. Markdown will be much cleaner. :D

Categories:

[ 0 comments | Add comment | Permalink ]

OpenSSL for Kirsle.net!


kirsle
Posted by Noah Petherbridge on Friday, April 18 2014 @ 06:18:01 PM

A while after the Heartbleed SSL vulnerability made headlines, Wired.com ran an article titled "It's Time to Encrypt the Entire Internet" urging everyone to deploy SSL/TLS encryption on their sites.

SSL certificates tend to be pretty expensive, though, which is one reason I hadn't looked into it that closely in the past. In a Reddit comment thread about that Wired article some people mentioned Namecheap as a good option for simple SSL certs. So, I got a simple domain-level certificate for $9 for Kirsle.net. :) So all kirsle.net URLs are now running over https! This blog post is about the experience of setting up SSL and wrestling with various applications in the process.

Generating the Certificate

The simplest guide I found that I followed to make a certificate was Generate CSR - Apache OpenSSL. One command creates a passphrase-protected key file, the next one generates the signing request:

openssl genrsa –des3 –out kirsle.key 2048​
openssl req -new -key kirsle.key -out kirsle.csr

You apparently need a 2048-bit RSA key these days before a Certificate Authority will consider your signing request. I pasted in my CSR file and filled out some forms, got an e-mail verification sent to the address on my WHOIS record for my domain, and before I knew it I was e-mailed a zip file containing my certificate and the Comodo CA certificates.

Certificate Chain File

Various apps will need your Certificate Authority's chain to be in a single file. You can create this file by catting the certificates into one file in "reverse" order, with your site's certificate on top, and the root certificate on bottom. Comodo gave me these files (and this is also the order for the chain file):

So I generated the chain as follows:

cat www_kirsle_net.crt COMODORSADomainValidationSecureServerCA.crt \
    COMODORSAAddTrustCA.crt AddTrustExternalCARoot.crt > cacert.pem

Apache2 mod_ssl

I'm running a Debian server, so I just symlinked the ssl.load and ssl.conf files from my /etc/apache2/mods-available into my mods-enabled, and then edited the ssl.conf. All I changed in it was to uncomment the SSLHonorCipherOrder on line.

I removed the sites-enabled/default-ssl and then edited my Kirsle.net config file to add a <VirtualHost *:443> version. I had to look at the default-ssl file to get an idea which options were needed (if I missed any, Apache would fail to start!)

Relevant SSL options for my VirtualHost:

    # SSL
    SSLEngine on
    SSLCertificateChainFile /etc/ssl/crt/cacert.pem
    SSLCertificateFile /etc/ssl/crt/www_kirsle_net.crt
    SSLCertificateKeyFile /etc/ssl/crt/kirsle.key
    SSLOptions +StdEnvVars
    BrowserMatch "MSIE [2-6]" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
    BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown

Note: if you leave out the chain file, web browsers will still behave fine (because they're smart enough to download the intermediary certificates themselves), but other things will break. For example, the Python requests module will throw an SSL exception if the server doesn't give it the intermediary certificates!

After making sure https://www.kirsle.net/ was working, I made an update to my Rophako CMS to support SSL sites better and then made the switch-over. Any requests going to my HTTP Kirsle.net are redirected to the SSL version and given a Strict Transport Security header.

As a fun side note, Apache supports Perfect Forward Secrecy by default (using the default SSLCipherSuite option of HIGH:MEDIUM:!aNULL:!MD5).

Starting or restarting Apache requires you to enter the SSL key's passphrase at the command line. For simple config updates, service apache2 graceful will reload them without needing a full restart, so you don't need to enter the passphrase then.

Dovecot IMAP

I use Dovecot for my IMAP mail server on Kirsle.net, and I wanted it to use my shiny new SSL certificate. Before this, I was using a self-signed certificate, and apparently Thunderbird doesn't even warn you if that self-signed certificate changes at any point. After the Heartbleed vulnerability was fixed, I re-generated new self-signed certs and was shocked that Thunderbird happily accepted the new certificate without even telling me. It would've been extremely easy to Man-in-the-Middle my e-mail server. (I had since then installed an extension in Thunderbird to police SSL certificates for me as a workaround).

So, configuration is pretty simple, just edit /etc/dovecot/conf.d/10-ssl.conf and enter in the new paths to your chain file and private key. Note that if you use just your domain's certificate, clients like Thunderbird that support SSL properly will complain about the certificate being insecure, and unlike web browsers, Thunderbird doesn't bother downloading the intermediary certificates itself.

One catch with Dovecot is that if your private key file is encrypted with a passphrase like mine is, doing service dovecot restart won't work. Dovecot will start in a way where it won't support TLS but will otherwise appear to function normally.

To start Dovecot with a passphrase, you need to run dovecot -p (as root) to start the service. It will prompt for your passphrase at the command line and then start up. The service can be stopped normally using service dovecot stop.

Postfix SMTP Server

This one I'm a bit upset about. Postfix has absolutely NO support for using a passphrase protected TLS key file! Even their official documentation states that the key file must not be encrypted.

That is so full of wtf. Postfix is a widely deployed SMTP server for Linux, and it has to use insecure, unprotected TLS key files. So, I'm still using a self-signed certificate for Postfix (and my Thunderbird add-on will tell me if this certificate ever changes, so don't get any ideas!). I don't send outgoing mail very often, anyway, and if I care enough I'll PGP encrypt. But, I'll be looking into an alternative SMTP server sometime soon.

Categories:

[ 0 comments | Add comment | Permalink ]

Goodbye, PerlSiikir!


kirsle
Posted by Noah Petherbridge on Sunday, April 06 2014 @ 08:19:16 PM
As of a few minutes ago, Kirsle.net is no longer powered by Perl. Instead, I've been working on a new content management system written in Python to replace it.

The reasons for the switch-over are numerous:

  1. The old Perl code was originally written for my previous version of my other project, Siikir, and the code was for an entire social networking type of site with lots of features, among which were Blogs, Photo Albums, and Comments (the three that Kirsle.net made use of). Kirsle.net didn't make use of the other features.
  2. The Perl code also had some memory leaks, which I tried for a while to eliminate but wasn't making much progress with. It was running as a FastCGI script, and the most notable side effects of the memory leaks were that my web server would randomly kill off unrelated processes, such as Minecraft servers or my XMPP server, because my index.cgi on Kirsle.net was sucking up so much memory. ;)
  3. Setting up PerlSiikir on a brand new server was an hours-long task. It needed a recent version of Perl, which needed a perlbrew installation done, and then a lot of modules needed manual installation. Seriously, look at my install notes. The new Python web app takes only minutes to set up.
  4. I like Python better nowadays than Perl. :)
And the best news of all is that my new Python CMS is open source!

I named the new project Rophako, because I was sitting at the Github "new repository" screen for a half hour trying to think of a name, and ended up just making use of my Azulian Translator to come up with a name. So, Rophako is Azulian for "Website." I'm a clever genius, I know. ;)

You can check out Rophako on Github: https://github.com/kirsle/rophako. The "default website" that comes with it isn't very polished yet; I literally just finished writing the code to support Kirsle.net. So, sometime later I'll tidy up the default website and have a working copy of it running on some subdomain like rophako.kirsle.net.

Anyway, this is the new CMS. I ported over all my old blog posts, comments, comment subscriptions, and things of the sort. All the old URLs should work too, due to my kirsle_legacy.py module in Rophako. If anybody finds any broken links or issues with the site, let me know. :)

Update (4/9/14):

I've polished up Rophako's default site and have an example running here: http://rophako.kirsle.net/

That's the site you'd get if you download and install Rophako (minus the blog posts and photos ;) ). So... the project is officially in "beta" status now and is usable!

Categories:

[ 0 comments | Add comment | Permalink ]

Exploring Grindr's Photo Cache


kirsle
Posted by Noah Petherbridge on Friday, April 04 2014 @ 01:21:03 AM
A long time ago, the Grindr for Android app used to store its photo cache on your SD card, but lately they hid them away in the app's private space to make them slightly more difficult to get to. I decided to go exploring using Root Browser and see what I could find out.

When I say "photo cache" I mean the place where Grindr downloads pictures locally so that it doesn't need to keep redownloading everyone's pictures all the time and consuming a lot of unnecessary bandwidth. Grindr caches both profile pictures and pictures received over chat messages. They both go into the same place. So if you have access to that place, you can get high resolution copies of all pictures received over chat and have them on your PC. :)

First of all, you'll need a rooted Android device for this, because the Android OS normally doesn't allow apps to get into each other's private data folders. The Root Browser app is a file browser that's root-aware (so it will prompt for root permission when you attempt to open a folder that ordinarily you can't open without root).

So, without further ado, Grindr's photo cache is located at /data/data/com.grindrapp.android/cache/picasso-cache/. This folder may contain a lot of files, mine had 3,458 and so Root Browser took a while to load that folder. You can copy it somewhere under /mnt/sdcard and then get to your files from a PC that way. Make sure the files are no longer owned by "root" when put in the SD card part, or you may run into issues when accessing them from your PC.

Most of the files in this folder have hexadecimal names that appear to be hashes of some sort, and the names usually come in pairs, one with a ".0" file extension and the other with a ".1", for example one I found on my phone was 4e21d675447678d0493bc8cb41a56e8d.0.

The ".0" file is a plain text file, and most of the ".1" files are the JPEG images. I use Linux, and my file browser automatically identified the types of all the files and showed thumbnail images for all the ".1" files. So, most of the time if you rename one of the ".1" files to have a ".jpg" extension you can see the images under Windows.

Some of the .1 files aren't images though. Some are more text files, and I peeked inside one to see what it was:

$ cat c1749deee81d4fece16d836e177c5852.1
[{"messageId":16970,"title":"Calling All DJs & Bartenders","body":"Are you one of the sexiest DJs or bartenders and able to work a paid event on the afternoon of April 27th in Palm Springs? If so, send us your information and a link to your website to palmsprings@grindr.com or simply tap 'More' to email us directly. ","actionTitle":"More", "dismissTitle":null, "expirationDate":1396853940000, "url":"mailto:palmsprings@grindr.com"}]

These appear to be the broadcasted pop-up messages shown in the app sometimes.

Now, the other interesting files are the ones with the ".0" extensions. These appear to be debug information, and they're basically the full HTTP request dump used to download the ".1" file. Here's what the one looked like for my profile picture (in case the Grindr CDN link stops working and you're curious, it's this picture):

$ cat 4e21d675447678d0493bc8cb41a56e8d.0
http://cdns.grindr.com:80/images/profile/1024x1024/d8dfd4eb2abd9c4d29653587cc87912b393bac97
GET
0
HTTP/1.1 200 OK
14
Accept-Ranges: bytes
Content-Length: 72057
Content-Type: image/jpg
Date: Fri, 04 Apr 2014 20:09:05 GMT
Etag: "98af07f8697f854734874296a90c640f"
Last-Modified: Sat, 01 Mar 2014 22:05:22 GMT
Server: ECS (lax/2851)
x-amz-id-2: [REDACTED]
x-amz-request-id: [REDACTED]
X-Android-Received-Millis: 1396642144430
X-Android-Response-Source: CONDITIONAL_CACHE 200
X-Android-Selected-Transport: http/1.1
X-Android-Sent-Millis: 1396642144347
X-Cache: HIT
I edited-out the "x-amz" headers because I'm not sure how secret those are supposed to be.

When browsing through my cache folder I also saw some pictures that weren't profile pics, but were sent over chat messages. These always seem to be the full resolution of the original pic sent, i.e. not thumbnails or anything. The ".0" file looks the same as for a profile picture, except the URL downloaded begins with "http://cdns.grindr.com:80/grindr/chat/" and the server headers respond with a "Content-Type: binary/octet-stream" (which causes a web browser to download the picture to disk instead of displaying it in the browser).

Some of the ".1" files are actually empty (0 bytes), and their .0 files indicate that these are the ad banners (requesting a URL from googleads.g.doubleclick.net). So it looks like whatever system in Grindr is responsible for downloading pictures also sorta deals with downloading ad banners, except it doesn't actually save the banner into the cache folder.

The last somewhat not-very-interesting file in the cache folder is called "journal", and it's a text file. By reading the first couple lines, it appears to be part of libcore.io.DiskLruCache, a bit of Java code that provides a rotating offline cache. This probably means that, if Grindr's cache folder fills up, it will automatically remove old files to make room for new ones, so it can keep its overall disk usage under control automatically. The journal file appears to list the hash names of all the other files in the folder, along with words like "CLEAN", "DIRTY", and "REMOVE".

Categories:

[ 1 comment | Add comment | Permalink ]

Skype and Windows Live Messenger


kirsle
Posted by Noah Petherbridge on Friday, February 28 2014 @ 12:17:26 PM
Back in the day, I ran a couple of chatbots on Windows Live Messenger (although it was called MSN Messenger then), so I'm reasonably familiar with how the Microsoft Notification Protocol (MSNP) works. We had a Perl module called MSN.pm which works with the MSNP10 version of the protocol, and it probably still works today.

That's right, the Windows Live Messenger protocol is still perfectly alive and well today. A while back, I booted my Windows OS on my PC where I still had Pidgin set up to sign me into MSN, and surprisingly it still worked. One of my Skype contacts sent me a message over Pidgin, and their "MSN e-mail address" had an "@SkypeDomain.fakedomain" domain part. It seems that now, though, while the MSN servers are still up, they at least block non-Chinese users from authenticating (Pidgin says "invalid response from server").

The Skype/MSN merger was done in a pretty half-assed way by Microsoft:

It appears that the Skype client actually acts like a "mini Pidgin": when you sign in with your old MSN account, Skype actually signs you in separately to the Skype and MSN servers. And, on the MSN side of things, the "@SkypeDomain.fakedomain" extension was probably implemented similarly to what happened when MSN and Yahoo joined forces, and your Yahoo contacts on MSN would have "@yahoo.com" domain extensions.

I don't get why Microsoft doesn't just pull the plug on MSNP completely, and force everyone to get a Skype name if they don't already have one linked with their MSN accounts.

Categories:

[ 0 comments | Add comment | Permalink ]

[ RSS Feed | Older > ]
Kirsle
» Homepage (RSS)
» About Me
» Photo Albums
» Guestbook
» Contact Me
Channels
» Linux (49)
» General (48)
» Perl (34)
» Rant (22)
» Software (15)
» HowTo (11)
» RiveScript (9)
» Minecraft (9)
» Gnome 3 (8)
» Android (8)
» Windows (8)
» Curiosity (7)
» HTML (7)
» Siikir (7)
» Design (6)
» Tk (6)
» Gay (5)
» Java (4)
» Blackhat (4)
» Reviews (4)
» VirtualBox (4)
» Ideas (3)
» DOS (3)
» Python (3)
» Photos (3)
» KAGE (3)
» Xfce (3)
» ttf2eot (3)
» Licensing (3)
Creativity
» 3D Renderings
» Flash Animation
» JavaScript
» Fonts
» Metacity
» Tutorials
Software
» RiveScript
» Error Generator
» Tk Calculator
» Terminal Apps
» CyanChat Client
Web Tools
» TTF to EOT
» Text Fader
» Favicons
» Distance Calc
» Azulian Encoder
» XBM Masks
Subdomains
» Shell Scripts
» Linux RPMs
» Rophako CMS
» Kirsle::Nano
» Minecraft Server
Miscellany
¤ Pokemon Fuchsia City
¤ DOS and Windows
¤ Raspberry Pi
Links
¤ Google+
¤ Facebook
¤ Twitter
¤ MySpace
¤ Github
¤ CPAN
Fan Club
» Log In