PSJailbreak USB Gadget kernel driver

On August 30, 2010, in Development, PS3, by kakaroto

Hi,

***

For those who don’t want to read a long post, here’s the summary : I’m trying to write a USB gadget driver to make my N900 act as a hub, I don’t know if I can get it to work because the kernel subsystem doesn’t seem to allow me to do it. If someone knows how to get a request’s destination address, or override the usb_gadget_ep0.c SET_ADDRESS, or knows of limitations that would prevent me from making it work, let me know. I also have ‘working code’ for the usb hub now, but it seems that when I simulate a device insertion, my computer’s (not the N900′s) kernel crashes, so I’m a bit stuck.

Read the rest if this article interests you.

***

Some of you already know about the PSJailbreak, for those who don’t, it’s a USB dongle that exploits the PS3 and allows you to run unsigned packages (homebrew).

Some people tried (and some succeeded) to create a ‘cheap’ clone of the dongle by reverse engineering what it does, and rewriting it into some ATMega microcontroller.

My idea was to use an existing programmable linux-based device (my N900) to act as the dongle. So I started looking inside the kernel’s source to understand how I can achieve that. I found that the kernel has a ‘usb gadget’ subsystem for writing gadget drivers (in other words, a driver to make your device act as a slave/peripheral) so I started writing a gadget driver.

I must say it wasn’t an easy task (for someone with ~zero kernel experience) especially considering that the only ‘real’ documentation I found was the undocumented source code of other gadget modules…

Anyways, the PSJailbreak dongle emulates a USB Hub with multiple devices getting connected/disconnected to it, so I tried to write a driver to emulate a USB Hub, I thought that it would be a great idea and useful, since it could be used in order to allow my N900 to be in PCSuite mode *and* mass storage mode at the same time, without having to make that annoying choice everytime I plug it into USB.

Anyways, I first realized that I can’t just insmod/rmmod drivers to emulate a device getting connected/disconnected, because the usb_gadget_register_driver doesn’t allow us to register more than one driver. Ok, makes sense, I can live with that, but this means that I’ll have to modify the kernel to make sure the usb_gadget_register_driver redirects to my hub’s code to simulate the insertion/removal and let my hub driver be the only one registered on the controller. Anyways, for my use case, I thought I can just write all the code for all these ‘virtual devices’ directly into my driver for now.

Second issue I came up with is that the drivers never get a SET_ADDRESS.. that’s handled internally by the kernel (drivers/usb/musb/msub_gadget_ep0.c) which means that even if I say “new device connected”, if the host sends me a SET_ADDRESS, I won’t get it, so I can’t map addresses to my virtual devices… but not only that, but I found no way whatsoever to find what is my current address, or to which address a message is being sent… I suppose it’s all being handled by the usb subsystem.. but I can’t find a “if (destination != self->address) return; anywhere in the code either.. which makes me think that it might be handled by the controller itself.. (since we do receive messages destined to other devices, if we’re connected to a hub, it has to drop those somewhere), but I don’t know, either the controllers don’t let me do what I want, or the kernel’s USB subsystem was never written to allow for USB hubs to be created. I figured that if I could at least simulate a device being connected, I should be able to find out how the kernel would handle the newly received SET_ADDRESS or the requests being received to the virtual device… then maybe I would understand a bit more how to do it and whether or not it’s even possible.

Call it bad luck, but now, whenever I plug my N900 (with my driver module loaded) into my laptop (linux debian, kernel 2.6.32-5), my laptop crashes.. it completely freezes up, the kernel panics, and then I’m forced to reboot it.. I’ve looked at what messages I’m sending/receiving from the N900′s dmesg (yes, the N900 is perfectly fine and doesn’t kernel panic), and I compare it with the USB dump of a generic hub being plugged into the computer, and I see no difference, I’m doing exactly the same! And yet, my kernel segfaults, and now, I’m stuck as I don’t know how to move forward.. I only got a partial stack trace, I know the khubd thread gets the segfault, and that it’s when it’s trying to build a URB…  there also seems to be some error being reported by the power/battery manager or something, so maybe it has something to do with bad/wrong values of self-powered/power needs of the device.. but that’s it…

I went to the #kernel channel on freenode, asked about this issue, asked how to get proper debug/stacktrace, and asked how a usb gadget can know its own address, but noone seems to care/answer/be awake. So that’s why I’m posting this on my blog.. first, to let everyone know what I’m doing and how advanced (or not) I am in the project, but also to ask people for help, if they know of a solution to my problem, let me know in the comments. Please, do not post comments like “I have a PS3/N900/something if you need help testing”… I don’t.

Finally, I’d like to finish by saying that I do not condone piracy. The PSJailbreak is an exploit that jailbreaks the PS3 allowing you to run unsigned code, it opens the door to homebrew and yes, also to piracy, but it’s each individual’s choice to either use it for legal applications or not. It is fair use to be allowed to make backups of your expensive games (and I’ve got about 50+ disc-based games). I’m doing this project only because I like the challenge, I thought it would be a good experience for me to dive a bit into the kernel code, and I found it entertaining. I also wanted to showcase the power of the N900 even more by making it become any usb device I want.. even a PSJailbreak clone, I don’t think anyone has used it in this manner yet.

Thanks for reading!

Update : I got a stacktrace from the kernel crash!

KaKaRoTo

Tagged with:  

18 Responses to PSJailbreak USB Gadget kernel driver

  1. tretle says:

    Could the n900 be the first platform with a softmod for the ps3 :D Keep up the great work, itching to get linux back on the ps3 :D

  2. [...] Post source… Related Posts:Doodle Blog » Blog Archive » Use the Doodle Gadget everywhereBlogUpp! | the blog: Meet Blogger gadget5+ Best places to Buy/Sell a Blog | Gadget CageTech Blog, WordPress Theme For Technology Gadget BlogsWhere Will Your Next Gadget Be Made? | Computer Nerds Blog [...]

  3. joerg DocScrutinizer says:

    >> there also seems to be some error being reported by the
    >> power/battery manager or something, so maybe it has something to
    >> do with bad/wrong values of self-powered/power needs of the device..

    The N900 will happily draw up to ~1A from USB, for charging, if you manage to fool charger detection in musb core USB driver. I’d not be surprised to find any desktop PC to freeze on that.

    You might try to use an actual (preferably externally powered) hub between your PC and the N900, to fence N900 possible VBUS current intake away from your PC

    HTH, good luck
    /j

  4. Maxximuscool says:

    Go KaKaRoTo!!
    This really the best news so far :)
    I missed my YDL6.2 on my PS3 lol.
    Bring back the good old time and hopefully we can get an emulator install inside XMB from this :D

  5. humble says:

    I second Maxximuscool.. woot woOT!

    Hey maybe you should ask titan off (maemo.org) he is one of the kernel dev’s(main) for N900 (i know you know) thanks anywayz.. for challenging your self. :)

  6. nafan says:

    I was thinking about doing the same thing on my old 770 tablet, considering the activated “USB Host” fonctionality could help…

    Any advice on how to start this port ?

  7. [...] KaKaRoTo's Blog Open your communications! « PSJailbreak USB Gadget kernel driver [...]

  8. kakaroto says:

    @tretle, yes, the N900 will probably be the first ‘device’ to support the jailbreak, but with the PSGroove release, those Teensy microcontrolers are the first ‘platform’ to support the jailbreak;

    @joerg: I liked the idea of using a hub in between, unfortunately, I don’t think it would make the exploit work anymore. Anyways, i’ve fixed it.. check my latest blog entry to see how.

    @nafan: it would probably work on the 770 without modifications, but I’m not sure.. and you don’t need ‘usb host’ mode since this works as a gadget, not a host.

    @everyone else: Thanks for the support! I don’t think I need any more help at the moment, I’m close to finishing it!

  9. [...] 02-09-2010 Some dude is trying to make use of the N900 as a “Jailbreaker” for the PS3: psjailbreak-usb-gadget-kernel-driver // Share| Comments [...]

  10. Dave says:

    And there was me thinking “I wonder if my N900 could do this, maybe I ought to have a go”

    The internet always beats me to it!

    Awesome work.

  11. all i want is to turn my ps3 into a full blown mkv playing movie machine

  12. Roberto says:

    Keep the old code that causes the kernel crash around, since you found a kernel bug. An unmodified kernel should not crash no matter what crazy thing you plug into an USB port, the same way it should not crash no matter what crazy thing enters through your ethernet plug.

    Under certain conditions you can have *hardware* failure, but a *software* crash means a software bug. You could even shutdown public machines (think those machines that can print your digital photos) just by plugging your n900 there! This must be fixed upstream, report it to linux developers or your distro bug tracking system, please.

  13. ihsangamerz says:

    i want to register my blog to homelinux.net,how and where?

  14. forcer says:

    ihsangamerz, homelinux.net is domain owned by dyndns.org.
    DynDNS is just a service to easily access dynamic DSL/Cable/FTTx connections which IP changes very often.
    That means KaKaRoTo hosts his own server machine(probably at home with home internet connection).
    Conclusion: you can’t have blog on homelinux.net without having your own web server machine and constant internet connection 24/7.

  15. alux says:

    would it be possible to jailbreak your ps3 if you had a usb dongle with a rj-45 plug in? (like one of those usb to Ethernet converters) and plug it into your computer running linux/ubuntu or a vm of ubuntu?

  16. kakaroto says:

    @alux: no, probably not because the usb donge to rj-45 is a device on its own and would just translate the data from ethernet into data being ‘transfered’ through usb.. it wouldn’t just proxy messages through.