Kirsle.net logo Kirsle.net

Tagged as: Curiosity

Google knows when you wake up?
September 26, 2015 by Noah

Something pretty odd that's happened to me on more than a couple of occasions:

I'll wake up in the morning, and then a couple minutes later people start sending me messages on Google Hangouts, so my phone's making a bunch of noise. If it's early enough in the morning, some of those messages will even say things like, "you're up early."

The thing is though, I didn't even touch my phone yet. I didn't even look at my phone.

Messages like this don't tend to wake me up, but instead I don't start getting messages until just after I'm awake.

Read more...

Tags: 3 comments | Permalink
Internet Protocol version 6
July 20, 2015 by Noah

Last weekend I was pleasantly surprised to discover that Time Warner Cable already supports IPv6 at my apartment.

They shipped me a newer cable modem/WiFi router combo device earlier this year as part of their plan to upgrade everyone's Internet speeds in Los Angeles. I didn't realize that this modem also supported IPv6, and it wasn't enabled by default.

Read more...

Tags: 0 comments | Permalink
Linux Desktop Remote Code Execution via File URI
March 27, 2015 by Noah
I've discovered a sort of "remote code execution" vulnerability that affects all Linux desktops, particularly Fedora and Ubuntu but most likely all desktop Linux distributions could be affected, except for maybe Arch or Gentoo with extremely customized installations.

First and foremost: this requires the victim to click not one, but two random links sent to them over Pidgin (or any other program that does URL auto-linking the way Pidgin does). So it's not exactly the most severe vulnerability, but I found it interesting nonetheless.

Read more...

Tags: 0 comments | Permalink
Skype switched to the MSN Messenger Protocol
December 9, 2014 by Noah

Today, Fedora 21 was released and I upgraded to it immediately, and decided not to install Skype this time.

Skype for Linux has been poorly maintained, going years between updates sometimes, and who knows what kind of unknown zero-day vulnerabilities are in there. On my previous installation of Fedora, Skype twice showed a weird issue where it replaced some of its icons with Chinese (or Korean, or something) symbols. I posted about it on Reddit and the Skype forum with no responses, as if I'm the only one who's ever seen this. Was I hacked? Maybe.

I took the latest Windows version of Skype and dumped all its icons trying to find these weird symbols but came up empty-handed. And I don't know of any way to pry icons out of the Linux binary of Skype. So... for now I just don't trust it.


In other news, I decided to Google the Skype protocol, and see what the progress is on people attempting to reverse engineer it, to be able to build an open source third-party Skype client (e.g. to have support in Pidgin). The Wikipedia article said something interesting:

On June 20, 2014, Microsoft announced the deprecation of the old Skype protocol. [...] The new Skype protocol - Microsoft Notification Protocol 24 - promises better offline messaging and better messages synchronization across Skype devices.

I wrote previously that the MSN Messenger service was still alive but it looks like it's the future of Skype as well.

The Microsoft Notification Protocol (MSNP) is the protocol used by MSN Messenger/Windows Live Messenger. I'm reasonably familiar with it from back in the day when I used to work on chatbots that signed in to MSN Messenger to accept their add requests and carry on conversations with humans.

MSNP is a plain text, line-delimited protocol similar to SMTP. There is some outdated documentation up through MSNP10 that we referenced in developing an MSN module in Perl. As an example, this is what you'd see going over the network if somebody sent a chat message to a friend:

MSG 4 N 133
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-MMS-IM-Format: FN=Arial; EF=I; CO=0; CS=0; PF=22

Hello! How are you?

The protocol consisted of command lines, typically three letters long (some I remember offhand: NLN--go online, FLN--go offline, BRB--set status to "be right back", RNG--request a conversation with a contact ("ring"), ANS--accept a conversation request ("answer"), and MSG--send a message). The RNG and ANS commands were invisible to the user in the official client, but allowed for some interesting behaviors in our chatbots (like immediately sending you a message the moment you open their chat window, before you even begin to type anything!)

It's interesting that Skype "downgraded" to the MSNP though, given that Skype's old protocol was an impenetrable fortress of obfuscation and encryption that nobody's ever managed to reverse engineer. Even third party clients like Trillian that had Skype support were technically using SkypeKit, a developer tool that allowed Skype to be remote controlled, but which kept all the proprietary bits a secret still. On the other hand, it makes sense for them to make Skype conform to Outlook.com and their other services that used MSNP, rather than upgrade all their other services to use the Skype protocol.

The last version of Windows Live Messenger used MSNP19 (or MSNP21 depending who you ask), so the new Skype protocol is just the next version of MSNP.

In Googling MSNP24 I found this site where efforts are already underway at reverse engineering it. This other site has a lot more details on the current status of the Messenger protocol.

It's only a matter of time now until Pidgin can natively support Skype accounts. It will also be fun to program chatbots for Skype. :)

Tags: 5 comments | Permalink
Minecraft Clones
October 22, 2014 by Noah

These are just some of my thoughts on things that a Minecraft clone should probably do differently to Mojang's (well, Microsoft's) Minecraft, since a clone would be starting from scratch anyway and would be in a good position to fix some of the fundamental problems in the original game that are impossible/unlikely to be fixed by Microsoft.

Some of the Minecraft clones I'm familiar with so far are Terasology, Minetest, and Michael Fogleman's Craft. So far, Craft seems to do the most things "right" (as far as my list of fundamental improvements on Minecraft go) such as only storing changes to the blocks to disk rather than whole entire chunks, but more on that in a minute.

Disclaimer: I'm not a game developer. I am a software developer and have a particular interest in the technology used in games, i.e. I have the knowledge to write code for most given game mechanics and spend countless hours thinking about these sorts of things. I've read a lot about the technical internals of Minecraft and have read some of the source code to these clones, so this blog post is full of my own educated opinions on things. I may, at some point, take on the challenge of creating a Minecraft clone of my own, but until then these are just the things I've been thinking about over the last several months.

Read more...

Tags: 3 comments | Permalink
Exploring Grindr's Photo Cache
April 4, 2014 (updated February 10, 2018) by Noah
A long time ago, the Grindr for Android app used to store its photo cache on your SD card, but lately they hid them away in the app's private space to make them slightly more difficult to get to. I decided to go exploring using Root Browser and see what I could find out.

When I say "photo cache" I mean the place where Grindr downloads pictures locally so that it doesn't need to keep redownloading everyone's pictures all the time and consuming a lot of unnecessary bandwidth. Grindr caches both profile pictures and pictures received over chat messages. They both go into the same place. So if you have access to that place, you can get high resolution copies of all pictures received over chat and have them on your PC. :)

First of all, you'll need a rooted Android device for this, because the Android OS normally doesn't allow apps to get into each other's private data folders. The Root Browser app is a file browser that's root-aware (so it will prompt for root permission when you attempt to open a folder that ordinarily you can't open without root).

So, without further ado, Grindr's photo cache is located at /data/data/com.grindrapp.android/cache/picasso-cache/. This folder may contain a lot of files, mine had 3,458 and so Root Browser took a while to load that folder. You can copy it somewhere under /mnt/sdcard and then get to your files from a PC that way. Make sure the files are no longer owned by "root" when put in the SD card part, or you may run into issues when accessing them from your PC.

Most of the files in this folder have hexadecimal names that appear to be hashes of some sort, and the names usually come in pairs, one with a ".0" file extension and the other with a ".1", for example one I found on my phone was 4e21d675447678d0493bc8cb41a56e8d.0.

The ".0" file is a plain text file, and most of the ".1" files are the JPEG images. I use Linux, and my file browser automatically identified the types of all the files and showed thumbnail images for all the ".1" files. So, most of the time if you rename one of the ".1" files to have a ".jpg" extension you can see the images under Windows.

Some of the .1 files aren't images though. Some are more text files, and I peeked inside one to see what it was:

$ cat c1749deee81d4fece16d836e177c5852.1
[{"messageId":16970,"title":"Calling All DJs & Bartenders","body":"Are you one of the sexiest DJs or bartenders and able to work a paid event on the afternoon of April 27th in Palm Springs? If so, send us your information and a link to your website to palmsprings@grindr.com or simply tap 'More' to email us directly. ","actionTitle":"More", "dismissTitle":null, "expirationDate":1396853940000, "url":"mailto:palmsprings@grindr.com"}]

These appear to be the broadcasted pop-up messages shown in the app sometimes.

Now, the other interesting files are the ones with the ".0" extensions. These appear to be debug information, and they're basically the full HTTP request dump used to download the ".1" file. Here's what the one looked like for my profile picture (in case the Grindr CDN link stops working and you're curious, it's this picture):

$ cat 4e21d675447678d0493bc8cb41a56e8d.0
http://cdns.grindr.com:80/images/profile/1024x1024/d8dfd4eb2abd9c4d29653587cc87912b393bac97
GET
0
HTTP/1.1 200 OK
14
Accept-Ranges: bytes
Content-Length: 72057
Content-Type: image/jpg
Date: Fri, 04 Apr 2014 20:09:05 GMT
Etag: "98af07f8697f854734874296a90c640f"
Last-Modified: Sat, 01 Mar 2014 22:05:22 GMT
Server: ECS (lax/2851)
x-amz-id-2: [REDACTED]
x-amz-request-id: [REDACTED]
X-Android-Received-Millis: 1396642144430
X-Android-Response-Source: CONDITIONAL_CACHE 200
X-Android-Selected-Transport: http/1.1
X-Android-Sent-Millis: 1396642144347
X-Cache: HIT
I edited-out the "x-amz" headers because I'm not sure how secret those are supposed to be.

When browsing through my cache folder I also saw some pictures that weren't profile pics, but were sent over chat messages. These always seem to be the full resolution of the original pic sent, i.e. not thumbnails or anything. The ".0" file looks the same as for a profile picture, except the URL downloaded begins with "http://cdns.grindr.com:80/grindr/chat/" and the server headers respond with a "Content-Type: binary/octet-stream" (which causes a web browser to download the picture to disk instead of displaying it in the browser).

Some of the ".1" files are actually empty (0 bytes), and their .0 files indicate that these are the ad banners (requesting a URL from googleads.g.doubleclick.net). So it looks like whatever system in Grindr is responsible for downloading pictures also sorta deals with downloading ad banners, except it doesn't actually save the banner into the cache folder.

The last somewhat not-very-interesting file in the cache folder is called "journal", and it's a text file. By reading the first couple lines, it appears to be part of libcore.io.DiskLruCache, a bit of Java code that provides a rotating offline cache. This probably means that, if Grindr's cache folder fills up, it will automatically remove old files to make room for new ones, so it can keep its overall disk usage under control automatically. The journal file appears to list the hash names of all the other files in the folder, along with words like "CLEAN", "DIRTY", and "REMOVE".

Tags: 22 comments | Permalink
Secure IM Concept
February 6, 2014 by Noah
I was randomly thinking a bit today about how to implement a secure, encrypted instant messaging system. One that could be so easy to use, that mere mortals wouldn't need to worry too much about public/private key pairs, and yet the system would be self-verifying and all very automatic.

The base concept I was thinking of would be that usernames on this system would be numeric, a bit like ICQ numbers. People apparently didn't mind being assigned some "random" number that they logged in with and gave out to friends. But, unlike ICQ numbers, the numeric IDs on this system would actually serve a purpose (instead of being simply the auto-incrementing primary key in ICQ's database).

Each user's client would generate an RSA public/private key pair, which is obviously the foundation for any sort of secure messaging platform. Then, you'd take the fingerprint of the public key (a short hexadecimal string, which could look for example like "45:2f:a5:d8:13:95:ba:03:51:c4:8d:ac:82:a8:4c:6a"), and you'd turn that into an integer number. Then you'd take, say, the first and last 5 digits of the number to create a 10 digit "screen name" number.

For the sake of continuing my description, let's pretend that the number came out to 567-8426-789. When setting up the chat client for the first time, it generates keys behind the scenes, and says "Your login number is 567-8426-789. You can give this out to your friends so they can add you to their buddy lists."

This idea stems from the general practice where, suppose you downloaded a Linux OS DVD, and you wanna verify the download so you look up the sha256sum for it. You're not going to compare the two SHA-256 hashes character by character; you'll check that the first 5 or 6 digits are the same, and the last 5 or 6 too. If those are, you can reasonably assert that your copy of the DVD is the same as the intended one. The numeric login ID of this secure IM network would be similar.

By giving out your ID number to a friend, they're automatically verifying your identity, since your ID is based off your public key. If somebody was trying to impersonate you (with a completely different RSA key pair), they wouldn't likely end up with the same numeric ID so it wouldn't work. ;)

Of course, the IM clients themselves would include features to do a more thorough verification of the friend you just added, like allowing you to see their full hexadecimal fingerprint, and also show a visual fingerprint to make it even easier to verify at a glance that your friend is who you expect them to be.

Anyway, this is just the basic idea I had. As far as syncing other devices to use the same account goes, it would probably need something like the old/current method Firefox Sync uses, i.e. needing to enter a "random" code from one device into the other so that your private key can be securely transferred between them without a third party in the middle being able to see. Alternately, the system would be federated (so you could host your own servers, like XMPP), and your local server may be trusted with an encrypted copy of your private key, encrypted using your log-in password. But these are all details to be figured out later.

Tags: 0 comments | Permalink
HTML::Defang
June 19, 2012 by Noah

I decided to take a serious look at the HTML::Defang module. It's supposed to take some arbitrary HTML input and sanitize it, removing anything potentially malicious in the process (attempts to execute JavaScript code, embedding of iframes, applets, etc.)

It does a pretty decent job, but I found one thing it doesn't handle very well by default. In CSS code, it will attempt to comment out an attribute if you attempt to use JavaScript with it. Some example input:

<span style="background-image: url('javascript:alert(1)')">

HTML::Defang will turn that into this:

<span style="/*background-image: url('javascript:alert(1)')*/">

But, if you begin and end your CSS attribute with an end-comment and start-comment instead, HTML::Defang leaves the code looking like this:

<span style="/**/background-image: url('javascript:alert(1)')/**/">

Now, granted, this sort of exploit only really hits Internet Explorer users (at least for older versions of IE), but it is a pretty big issue still. This is basically how Samy pwned MySpace, after all.

Anyway, I've written a test CGI script for HTML::Defang: you can try to break it here. I added a custom CSS handler that will neutralize JavaScript attempts from the CSS code to handle that problem I found in HTML::Defang. You can see the source code by clicking the link at the bottom of that page.

If anybody finds a way to get JavaScript to execute on that page, let me know. :) I've tried all the usual tricks and haven't found a loophole yet.

Tags: 1 comment | Permalink
XMPP Gateways
June 13, 2012 by Noah
Today I decided to look into how well XMPP gateways work. Here's how it went.

XMPP Gateways (or, alternately, Transports) are services for XMPP that let users sign on to other instant messaging networks (AIM, MSN, YMSG, etc.) through their XMPP account. So... you sign in to XMPP and the server signs you in to your other networks, and you can do all your chatting through just the one XMPP connection (Wikipedia has a nice diagram of this).

So I was wondering whether this would be a good solution for programming chatbots that work on many different IM networks. In Perl, the options for IM network connectivity aren't all that great at the moment; there is Net::OSCAR for AIM, Net::XMPP, and that's about it. Matt Austin and I were working on Net::IM::YMSG for Yahoo Messenger, but it's not that great yet (it can chat and accept add requests, but after accepting you don't see it online until the program restarts). For MSN Messenger, there's a dusty old MSN.pm module somewhere on the RiveScript forums that might still work but hasn't been updated in years.

I'm not very motivated to work more on Net::IM::YMSG because YMSG is dead to me. I created a new Yahoo ID recently, and made sure to opt-out of any directory listings, and it still gets add requests by spam bots on a regular basis. The ID is literally listed nowhere. I don't know how they find me. It is my opinion that nobody should take YMSG seriously anymore. I don't know anybody who uses it, so developing new code to use YMSG isn't worth my time.

With XMPP gateways though, one could just write an XMPP bot and let the gateways do all the work. Then you can have a bot sign on to AIM, Yahoo, MSN, ICQ, IRC, and even some obscure networks like MySpace IM, without needing to use native libraries for each protocol yourself. Sounds great, but I had some questions about how well gateways work (how many features they support per network, etc.)... so the best way to find out was to test it!

I installed the Openfire XMPP Server on my web server. Installation was surprisingly super easy. Basically all I had to do was this:

# apt-get install sun-java6-jre
# dpkg -i openfire_3.7.1_all.deb

This installed it and automatically started it, and then I went to the admin panel port on my server to set it up. I didn't have to touch any config files or anything; everything was done through the web panel and was dead simple. I set up a Jabber user @kirsle.net and then I went out in search of a gateway plugin.

I found Kraken to fill the role. Installing this plugin was super ridiculously simple too, I just copied kraken.jar to /usr/share/openfire/plugins and Openfire automatically saw it and extracted it and a new "Gateways" tab appeared in the web panel. Black magic. I later discovered that I could've just uploaded kraken.jar to a file upload field on the Plugins page on the web panel, too. This is, by far, the easiest server software for Linux that I've ever seen. I can't believe Openfire is free and open source.

So after setting the gateways up for the protocols that I care about, here is what I found:

  • AIM and MSN support buddy icons, but Yahoo doesn't.
  • AIM does not support AIM chat rooms or file transfers. Presumably this is the case for the other gateways as well.
  • AIM profiles aren't supported. The gateway sets the AIM profile to "oscargateway".
  • When my Yahoo gateway signed on, I got 5 add requests from spam bots. Denying these in Pidgin apparently didn't "stick", as the 5 requests came back the next time Yahoo signed on too.
  • Chatting with AIM users over the gateway shows their screen names as "screenname@aim.kirsle.net". This is good. I haven't seen how MSN names are formatted though.
  • The MSN gateway was a bit buggy. It doesn't sign on right away. It took many many minutes before it finally signed on. I didn't test it very thoroughly.
  • I couldn't get my ICQ gateway to sign on. It said wrong password but I'm sure I had the correct one.
In short: it appears that only basic chatting and buddy list management is supported. For Yahoo (and presumably MSN), add requests from other users seem to come in just fine. Presumably you would be able to accept the add requests as well. Multi-user chat rooms and file transfers don't work. I haven't tested Attention/Buzz/Nudge features.

For simple chatbots that only need to communicate over IM, XMPP gateways should work out fine. But if you have a native way to connect to a particular network, that method would most surely be preferred.

The reason I like the @aim.kirsle.net etc. suffixes on gateway usernames is because it would make it easier for a chatbot to distinguish unique users across multiple networks. In my Aires bot (which supports AIM and YMSG, through Net::IM::YMSG), and in all my older bots from years gone by, the protocol-facing code would format a user's name like "AIM-kirsle" or "YMSG-kirsle" before passing it to the main bot code. This way I could make an AIM user an admin for my bot, without any confusion about overlapping Yahoo IDs of the same name. Since the XMPP transports make a unique subdomain for each network, this can be used instead.

In conclusion, I think it will be worth it to add XMPP support to my Aires bot so that transports can be used for people who want them. Native support for a network is always preferred, but with transports you can easily put your bot on a lot more networks with very little effort.

Tags: 1 comment | Permalink
Sequence Breaking
October 30, 2011 by Noah
I spent most of the day today toying around with Pokemon FireRed Version - trying to sequence break it, mostly.

Screenshot

Originally, I wanted to see what would happen if you got to Fuchsia City without having taken one of the fossils from Mt. Moon... because in the Fuchsia City Zoo, there will be either an Omanyte or a Kabuto there -- the opposite fossil from the one you chose early in the game. So I wanted to see what the game would do...

But then I realized that, Fuchsia City is the perfect city to start the game in because, while there's a steep difficulty curve to get your bearings, you can actually continue the game from there and get back on track. This can't be said of all the other cities (especially Cinnabar Island).

I've divided my whole quest up into four parts, complete with screenshots. Read about my sequence breaking quest here.

Tags: 0 comments | Permalink