Introduction to Cryptocurrencies

On March 27, 2014, in Educational, by kakaroto

Tips :

BTC : 15fXq5FzKaUArzQ8zBWfJADMn1qTQ5w5Y6
DOGE : DK4uNKX99VTgEv8BZdeepsYJHKhev3oR1j

Hello everyone!

Today, I received an email from a friend saying that he knows that I’m into crypto currencies recently and he wanted to know if I could give him some pointers… in true KaKaRoTo fashion, I wrote him a long email to explain everything there is to know about cryptocurrencies. I think there’s a lot of interesting stuff in there that others might find useful so I’ve decided to make it into a blog post.

Note that I didn’t edit the email, so read it as an “email to a friend”.

Concerning crypto currencies, it’s a whole world and really quite interesting. I’ll try and give you as much info as i can. Note though that everything I say below may not be 100% accurate, as I might say something wrong, either because I misunderstood it myself or maybe to keep things simplified.
I don’t know how much you know about it, so at the risk of saying things you know, I’ll just assume you know nothing about cryptocurrencies.
So cryptocurrencies started with bitcoin in 2010 i think, satoshi nakamoto (an anonymous name of person or a group) released a white paper explaining the concept and created the currency, wallet, website, etc.. Of course you can check bitcoin website for more info, but the concept is basically that btc have a value, which is defined by how much people want to pay for it, but as is, it’s just a number. The system works by constantly generating coins until a maximum is reached, the concept follows gold mining where as long as you mine, you get more of it but there is a limit on the resource in the world and the value of gold depends on how much people want to pay for it, but other than that, it has no real value (it’s just a metal, right?).
I’ll explain about btc (bitcoin) then expand on the other cryptocurrencies (that we call altcoins).
Btc has a blockchain which is a public ledger which is made up of “blocks”, each containing transactions, in order to create an account you just generate a private and public key, the public key is your “account number” and the private key is your wallet. To send BTC from one person to another, you create a transaction containing your origin and destination accounts and the amount then sign it with your key then post it on the network which will add it to the current block, other nodes in the network will then check that the transaction was signed with the origin account’s private key, and they will sign your transaction in turn in order to confirm that it’s been verified.
At the same time, you have “miners” which are trying to generate the next block, more on that in a second. When a new block is generated, it gets added to the blockchain (ledger) and all new transactions get written into this new block. The previous one gets finalized. To ensure security of the network, new blocks are constantly being generated, for bitcoin it’s set to generate a new block every 10 minutes approximately. So, the block get generated by miners, to do that, they need to prove that they worked for it. The proof of work is based on a hashing algorithm, basically the take the current block header’s hash, they add random data to it then they make a hash of the data. I suppose you know what a hash is, so i won’t explain that. Basically the hash result must be below a specific threshold, if it is, you found the new block, if it’s not, you need to search again. So imagine a sha1 hash where the first 10 bytes are 0×00000000000000000000 you must be extremely lucky to find such data that gives this kind of hash. Well that’s what the PoW (proof of work) is based on. You keep hashing millions of random data (which include the hash of the current block) until you find such a lucky hash that is below the threshold, thus proving that you did work hard to find the new block. You can see the blockchain here for the current block (at the moment) for example : https://blockchain.info/block-index/381396/0000000000000000d0f673f0241c7aca3f2453b165a2f70014362733e0daad81
you see in the top-right where it says hash/previous block/next block, you see all the 0000 it starts with. That’s the block’s hash which is below a specific threshold. The reward for finding such a hash/block is that when you create the new block, you will add a new transaction to it, the first transaction of a block is always a transfer of 25 btc from “nowhere” into your address. That’s your block reward. You can see it in the blockchain link I just gave you, it has all the transactions, and the first one has “no inputs” and has 25.07 BTC (if it shows $ value, click on the green button below the value to show it in BTC). So there you go, that’s how you mine coins and generate new ones. Now the thing is, what is that threshold, and what happens if you can’t find the hash. Well, the threshold is called “difficulty” in the crypto world and it’s automatically adjusted after every block (or every 10 block or whatever the currency creator decided when he made it), and it’s based on the average time needed to generate the block. So let’s say you have 10 miners, each with 1GH/s comutation power, so the network has 10GH/s and for a difficulty of “5″ (let’s assume it means first 5 bits are 0), it takes an average of 10 minutes to find the hash. Now 100 new miners join the network with 10GH/s each, that’s 1000GH/s more to the network and the total network hashrate is now 1010 GH/s.. it will be a LOT easier to find that hash, so now it only takes 30 seconds to find it. But BTC spec says one block every 10 minutes, so the difficulty will increase to let’s say 13 to account for all the new hashing power, and now the hashes are found every 10 minutes.. some miners leave the network, difficulty goes down, etc… Of course, it’s not exact, it’s based on luck, but it’s “how probable that the next hash will be found in 10 minutes considering the current hashing power of the network”, sometimes with the same hashrate and same difficulty, it takes seconds to find the next block, sometimes it can take hours.. and yes, if it takes hours, then it takes hours, there’s nothing you can do about it, you just wait until it finds the block, then you lower the difficulty for the next one. No one actually sets the difficulty, it’s decided upon by the entire network. Everyone runs the same code, so everyone follows the same rules and agrees with each other. If for example someone doesn’t, then his hash/transaction/whatever will not be confirmed by other miners and it’s rejected. If two people find the next block at the same time, then one will get orphaned and the other will get confirmed, not sure how that works, but there’s some race condition/concurrency protection in the way confirmations are done. The same applies for transactions and accounts, if you send money to someone but you didn’t have the correct private key, then it won’t get confirmed by anyone else and the transaction is useless. That’s why whenever you do a payment or transfer, any respectable site will tell you they wait for X confirmation before unlocking the funds for example, it’s usually 6 confirmations, which can take a few minutes, it depends on the network and the hashrate and number of miners, etc…
So.. When you have an account, your private key is your wallet, and if you lose it, then you will have no way of signing any transaction for that account, meaning that the money is lost forever. that’s why it’s always very important to make a backup of your wallet.dat file somewhere safe, or to write it on a piece of paper, or something like that.. a lot of people have lost millions of dollars because their HDD failed and they didn’t have a backup of their wallet.dat. One even just threw it out because he thought it was useless, back when 1BTC was valued at 0.0001$.. and then when 1 BTC became 1300$, he regretted it..
Since BTC has a public ledger (the blockchain) and transactions are confirmed by the peers, and no one owns the network, then obviously, you can see the balance of any account you want (see previous blockchain and click on any address to see its full history and balance), that’s why some people will create a few new accounts (just generate a private key locally) for every transaction and split their funds through multiple accounts, this way someone seeing a transaction won’t know which of the destination is the one being paid and which one is the new account of the account holder. It is sometimes suggested to use one new account every time you make a transaction.. but I don’t really do that myself.
Now one last item, we talked about mining, but mostly about what we call “solo mining”, which is having your CPU or GPU calculating hashrates until it finds the right one and then you ‘win’ the 25BTC reward. But if you did that for real, you would never win it considering how many people are on the network, so what people do is use “mining pools”, which is basically a service that will send you much smaller computations to do and you give the result to them, and everyone joins the pool. When the pool is the one that finds the block, it will then share the reward proportionally with every miner depending on how many “shares” they sent.. so for example, here’s one of my shares for BTC in one of the pools I’m in :
Block          Value                     Status          Duration          Hash Rate         Your Shares                        Payout
292742      BTC 25.1399      43/120       13 minutes      10.02 Ph/s       5456/5000002904       BTC 0.00002743
So, you see the block id which yielded 25.1399 BTC, it has 43 confirmations (out of 120 required before the block is considered accepted/not-orphaned), it took 13 minutes to find it, the pool’s hash rate is 10.02 Ph/s (my rate is 11.02 GH/s), I sent 5456 shares out of a total of 5000002904 from the entire pool, and the payout is my portion of the 25.1399 BTC that was paid to me (yes, quite small for 11GH/s of hashing power, but consider the 10Ph/s of the pool… FYI, the network has 50PH/s).
Without mining pools, you wouldn’t be able to get anything.. I mean, sure I could try to find the block, and maybe I could and if I do, I’d win 25 BTC which is a LOT of money, but considering how huge the network is, it might take me years to find the block, or maybe I’ll never do.. so you join a pool and you share your luck with others. There are a few reward types a pool can use, either payout proportional to how much you contributed to that block, or proportional to the number of shares you sent in the past X minutes when a block is found (so if you’ve been on the pool for hours then you leave just before it finds a block, you still get something). anyways, not important.. the important thing to know is that if the pool isn’t the one that finds the block (there are a LOT of pools) then you don’t get anything. You can see the various reward types here : https://litecoin.info/Mining_pool_comparison#Reward_types
Oh and another thing, at specific blocks, the reward gets halved.. BTC started with 50 BTC reward, then at 100000th block, it became 25 BTC, it keeps getting halved until some point because, just like gold, the rarer it becomes, the harder it becomes to mine it. and the graph of number of coins in circulation should plateau towards the max, now jump right into it.

Anyways, now that the technical is out of the way, let’s talk about the theory of mining.
Mining bitcoin is impossible, not at home anyways.. usually a latest gen GPU will give you a few hunderd MH/s hashing SHA256, but GPU mining is so 2011, now everyone is using ASICs (Application-specific integrated circuit) which do a few TH/s easily. So any coin that is SHA256 is basically impossible to mine at home.. you would probably get about 0.01$ USD after a year of mining.. which would be less than your electricity costs. That’s where Litecoin (LTC) came into action! LTC uses Scrypt algorithm instead of SHA256 so the ASICs don’t support it, so it’s pretty much GPU exclusive, yeay! There are TONS of altcoins though, each with their own rules (how much reward per block, how much time per block) and their own specs (current difficulty, current exchange rate) which will be more or less profitable for you. The reference for me is coinwarz (http://www.coinwarz.com/) which I check every day because one very profitable coin today might be crap tomorrow. You can put your hashrate there (and electricity cost and power consumption of your GPUs) and it will tell you how much you will gain per day if you mined the coin. The problem is that Scrypt ASICs have just been released last week, so we’ll have people using ASIC for scrypt based coins soon… but they’re not that good, I mean, a 200$ ASIC gives you about 350KH/s which is slightly higher than a 200$ GPU which would give 300KH/s, but the nice thing is that it uses 2W of power, instead of 200W or whatever your GPU consumes.
There is also a new algo (kind of) called Adaptive-N Scrypt, which is just scrypt but with one of the constants made a variable (I think) which will make it hard to do an ASIC for it because memory requirements will increase everytime to prevent ASICs from catching up technology-wise. There’s also Keccak algo, but that’s only used by one coin, it’s called maxcoin and was released by Max Keiser, a financial journalist for RT (the tv channel). It was great, I made over 200$ with it pretty fast then value dropped so low that it was worth less than 20$.. thankfully, it went back up a little and I sold them and made 40$ back and lost 2 weeks of mining… if only I knew it would crash, I would have sold it when my balance was worth 200$.. but that’s part of playing the game! :)
This is another lesson, if you want to make it profitable, you mine something and you sell it right away into BTC (which is more stable).. I mined maxcoin because it was by far the most profitable, giving me about 20$ per day, but back then the value of one MAX was 0.01 BTC.. then it kept going down until it reached 0.0001 BTC. I managed to hold off selling until it started going back up then I sold it at 0.0004 just before it started dropping again, but if I had sold my mining revenue every day, I would have made a lot more money from it.
On the other hand, some things, you want to keep, for example, there’s Auroracoin (http://auroracoin.org) which is a coin that was created for the icelandic people who have huge financial issues and where 50% of all the coins are pre-mined and will be distributed to all the citizens of iceland, so I mined it and I thought that it would be awesome to have coins from this currency which might be widespread in an entire country.. but a couple of days ago they did the airdrop (where they allow icelandic citizens to claim their coins) and the value dropped..
Actually the value fluctuates depending on supply/demand.. if a lot of people are selling, then the price drops because you compete on the price in order to sell yours first.. if people want to buy, then price goes up. I suppose what happened with auroracoin is that people got their coins and just sold them in exchange for BTC since there was no infrastructure supporting AUR. But maybe in the future, it will start being accepted by merchants in iceland and people will start buying it and price goes back up. at least that’s the plan. Note also that it was basically a “free money for everyone” which goes against the whole idea of proof of work to get reward.. what do you do when you get free money? Note that there’s also now a SpainCoin and a GreeceCoin following the same 50% pre-mined for citizens principle, and I’m mining those as well (got 4 AUR, 20 GRCE and 68 SPA (sold 15 of SPA already when it was very high price)..)
There are also others than you want to hold onto because you know the price will go up, like for example, the very much liked and meme-based DOGECOIN! wouhouuu! ok, I do like dogecoin because it’s so popular. The dogecoin currency was created only as a joke by two guys, they never thought anyone would care, they made a logo using the shiba inu dog and used that meme as a base for the coin “wow, much coin, very transaction, etc..” and then they were wtf-ing when people actually started using it.. turns out it became extremely popular and value is sky rocketting. Only problem is that since it was a joke, all the coins will be mined in 10 months or so. but difference is that there is no limit on the amount of DOGE as opposed to other currencies, it will just become really small reward after the limit. The reward was random between 10 000 and 1 000 000 DOGE per block with a block time of 1 minute. Right now, I think they changed it to become a fixed amount because people were abusing the system by only mining doge when the next reward was the highest (since it’s a consensus, remember, it means the ‘random reward’ is not random at all, it’s based on an equation using the previous block’s hash as seed), which caused increased difficult on specific blocks and honest miners were only getting the small rewards. Anyways, it’s been halved once already so right now, it’s 250 000 DOGE per minute, and the price is 0.00000102 BTC. oh and yes, most exchanges are BTC to altcoin or altcoin to BTC, sometimes to/from LTC as well, then it’s USD/CAD/WHATEVER$ to/from BTC or LTC. so yeah, you do the math from your altcoin to BTC then according to today’s value of BTC, you know your altcoin’s worth in $.
So anyways, what’s special about DOGE is that it’s very popular, reddit mostly is making it the altcoin of choice, you always hear about it everywhere, like during the US regulations talks, they would talk about BTC and LTC (as the main coins) and they mentioned “or a coin based on a dog meme”. Doge is used a lot for tipping, charity, and all that, so you see a lot of causes evolving around doge, for example they raised 25000$ to get the jamaican bobsled team to go to the olympics, there’s doge4water (http://doge4water.org/) and just recently (last week actually, and funding finished yesterday) there was doge4nascar where they raised 50 000$ to sponsor Josh Wise car in NASCAR racing. http://www.doge4nascar.com/ and since they raised it, it’s been on every news outlet ( Fox news, the guardian, etc.. https://www.google.ca/search?q=dogecoin+josh+wise&safe=off&tbm=nws ) which is giving it a lot of exposure and popularity.. and just imagine that car with dogecoin logo on national television during nascar.. this will cause people to get interested in dogecoin, and to BUY dogecoin, which will cause the price to go UP! Also, in 30 days, the reward will be halved to 125 000 DOGE, which means the value kind of has to double in order to keep miners interested in mining dogecoin for profitability.. think about it, reward gets halved, means miners get half as much.. so they will stop mining it, if value doesn’t double, then they won’t be get enough and other coins will be more profitable.. of course, other possibility is that miners leave and the network hashrate (and so the difficulty) drops so those who remain get twice as much as before == same profitability… anyways even though dogecoin value has been dropping a lot lately, it is bound to go back up. It had already gained 10 times its value by the time I started mining it, unfortunately, I only had very little mined at that time (now I have around 145 000 DOGE in my wallet). On the other hand.. as someone recently told me “we don’t really see many parodies of gangnam style anymore” so maybe this hype around dogecoin is a bubble about to burst.. you never know!
That being said.. it’s a game, like stock market.. you “buy” (in this case by using your GPU’s time and electricity) coins and hope it goes up.. if you don’t hope that, then sell them right away.. then move away to the next coin, etc..
one very interesting thing is at coin launches! What happens in a coin launch is that there’s pretty much no one around, so if you get in just at the right time, you can get a lot really quickly. like for example, my auroracoin, I mined it too late.. my friend told me about it and I ignored him.. difficulty was maybe 100 or 200 (don’t know what it represents exactly, but it’s not bits of 0s) and he made a few AUR in a day on his home PC GPU (a cheap gpu which gives 60KH/s) then people got “wow” over it and started mining it, the value of it jumped and difficulty became 6000 (which is HUGE for scrypt based coins) so I started mining it and only got like 2 AUR after 5 days of mining on my mining rig of 2.1 MH/s… and my friend got “rich” quickly.. he sold and bought himself a new GPU to mine some more… So the idea is to find coins that will have a lot of exposure and impact, that people will like and you mine them from the very beginning before everyone else joins. Sure, you’ll still be a drop in the ocean.. but a 2MH/s drop in 200MH/s is better than a 2MH/s drop in 60GH/s network hashrate :) You can see the network hashrate on coinwarz by the way.
So one interesting coin coming up is H2O https://bitcointalk.org/index.php?topic=494229.0 it looked good a lot of people wanted to jump into it, the launch was meant for march 24th, but they had bugs and delayed it.. rumor is that it’s going to be this friday.. if you want to mine it and sell, then that might be my first choice. Second choice would be the spaincoin or greececoin because their difficulty is really low so you can make a lot of them really fast, and if they get adopted, their value can be worth a lot, but only in the future.. also it could crash and value becomes 0.. and of course, dogecoin! but it might be better to buy dogecoin, its price is very low right now, 102 (meaning 1 DOGE = 0.00000102 BTC) and I think it will go to 200 in less than a month.. and after nascar ( in two months) it could double again. You could also mine it, see how much you can make according to coinwarz. Be aware you need to find a good pool, if you use a small pool, there’s a chance it won’t find blocks and you won’t get anything.. a good pool finds blocks more easily, but the pool’s hashrate is obviously higher so you get a smaller share, but often.. a smaller pool will give you a bigger share, but less often, your choice which way to go.
Another way to look at it is that once ASICs start selling (There’s already one ASIC on the market, but the “Titan” asic is planned for Q2/Q3 of 2013 and is supposed to be massive), and asics take over the scrypt network, then you’ll see a LOT of people with their GPU mining rigs having to shift their focus, either by selling their rigs (so expect cheap GPUs soon, by the way AMD just dropped their prices yesterday on newegg.com (not .ca)) or by making them mine a “asic-proof” coin.. and that’s where Scrypt-N comes into play.. my prediction is that soon everyone will move to scrypt-n coins (spaincoin switched recently from scrypt to scrypt-n in an update) which means the difficulty for scrypt-n coins will go through the roof. and Vertcoin is the one who stated it all, and is probably the scrypt-n coin with the highest difficulty and highest exchange value. My prediction is that when all gpus go to it, its difficulty will go up a lot and by consequence, the rewards will get smaller (harder to mine), which means that for it to be profitable, its value has to increase, so it will probably have a big increase in exchange rate (it already seems to be increasing steadily along with its difficulty).. so the vertcoins that I mined now (relatively easily, 5 VTC so far) will be worth a lot more in a few months (or a year).

You can go at this two ways, the hoarder mode with hopes that in 2 or 3 years, your coins will be worth millions, or a seller who will sell coins as soon as he mines them to make a profit right now.. or you can try to play the market, predict increases and decreases and all that.. You could switch from one altcoin to another, or concentrate on only one…

Now the last thing about this “theory” section is about exchanges.. obviously, altcoins can be exchanged for BTC and BTC for CAD or USD. and for that, you use exchanges, the most known one for BTC was MtGox which fucked up and closed and is under investigation and all that.. there are a few who had issues, but the one I use and most people seem to use is cryptsy.. there are other well known exchanges like Bter, bitstamp, mintpal, etc.. The way it works is that it will generate an address for an altcoin just for you, you transfer money to that address and it counts as a deposit into your account, then you can trade (sell/buy) and you can withdraw the money afterwards.. exchanges will have a balance for each of your altcoins so it kind of counts as a bank, but it’s not recommended to keep your money in anything other than your own wallet (and secure your wallet.dat). MtGox lost millions of $ and many exchanges got hacked and lost people’s money. Same rule goes for pools, when you create an account in a pool, first thing you do is set your address for transfers and enable auto-payout, you don’t want to have the pool hacked or shut down and all your mined coins still in the pool.
So, I suggest cryptsy for most of your stuff, I like it.. but while it supports a lot of currencies, it doesn’t do all of them, so you have to use others from time to time.. for example, to sell my SPA (spaincoin), I had to use Mintpal since cryptsy doesn’t do SPA. For GRCE, I only found bittrex and cryptorush that support it (for now), etc..
Remember when I said mining new coins is very profitable.. one issue though about new coins is that they will start with a high value on the market, let’s take GRCE (greececoin) for example, when it started it was valued at 0.008 (BTC per GRCE) and I was mining it, great.. now my first problem, the rewards I mined are unconfirmed in the network (to avoid an attack of the network if someone has 51% of the network hashrate, you need to wait 120 blocks after the current one before you can use coins generated in that block), so I still can’t use them.. need to wait for them to be confirmed.. by the time they were confirmed, value dropped to 0.003.. but now I can’t sell my coins because first, not many exchanges support it.. actually, only one at that time, secondly, no one wanted to buy it.. it’s still new, no one is interested, the only people who know about it are those who follow the “new altcoin announcement” threads on bitcointalk forums, and those are already miners who mined a lot of the coin and who also want to sell it, they don’t want to buy it… so if i try to sell it, it won’t work.. and when no one wants to buy and you want to sell, what do you do ? well, you sell your coins at a value lower than the market price, so as soon as a buyer comes in, he’ll buy your coins, not someone else’s.. and all the miners fight over that by lowering the market price.. by the time buyers are in, the price dropped 10 times.. and currently, it’s at 0.0001, so.. 80 times less than the initial market price. All you can hope for is that this new coin will be successful and people will like it enough that the price will eventually start going back up, and you are left with a lot of coins mined during the launch.

Ok, I think you got the theory, so now let’s talk about practice! There’s not a lot to say, you need a GPU (forget about cpu mining), it needs to be an AMD, because Nvidia SUCKS.. you can look at the hardware comparison to get a good idea : https://litecoin.info/Mining_hardware_comparison
You can use your own hardware like my friend did until you earn enough by mining to buy more GPUs, or you can buy a dedicated rig, which is what I did (and also use my desktop GPU for mining when not in use). For your info, if you buy a rig, it’s better than just buying coins because once it pays for itself, then it will keep generating free money.. but most importantly, at the end of the day, if all your coins are worthless, you still have hardware that has resale value. BUT whatever you invest in this, you have to be prepared to consider that money as “lost”.. in other words, don’t spend money you can’t afford to lose.
For reference, I “sacrificed” 1500$ and bought a rig consisting of :
Power supply + Motherboard + CPU + RAM
2 Asus AMD Radeon R9 290
The power supply is quite important because your rig will use a lot of power, and the motherboard, you’ll want one with as much PCI-E ports as possible.. mine has 4 PCI-E 16x and 2 PCI-E 1x so I could put 6 GPUs on it.. problem is that a GPU uses two slots because of their width, and I can’t fit 4 in there.. and that’s why you can buy PCI-E risers (look for it on ebay). Anyways, with 4 GPUs, you’ll most likely need a 1500W PSU.. what I did was buy 2 PSUs of 750W because then you can power 2 GPUs with one PSU and the other 2 with the second GPU and just force the second PSU to be always on (by shorting PS_ON with ground on the ATX connector). I bought two because PSU was on sale and even though I only need one, I thought might as well take the second one for half price :p
The GPU was on sale as well, normal price is I think 660$, I got mine for 540$.. now on the US newegg, price dropped to 430$ from what I saw yesterday.
I chose the R9 290 because a friend of mine said they are better than similar KH/s cards in terms of power consumption. I tweaked it until I got 860KH/s per card which is not bad.
I didn’t need an HDD because I used linux running off a usb stick. If you chose to install windows, you need an HDD though..
Here’s a nice guide which I followed : http://www.cryptobadger.com/build-your-own-litecoin-mining-rig/ that website also has other nice articles if you want to read through them.
So I used the SMOS distribution which you just write to a usb key (no install) and you boot it and it starts mining right away, you’ll just have to edit the config file to point it to your own pools : http://www.smos-linux.org/
And it gives you a web access similar to this : http://bamt.webboise.com/mgpumon/
Only issue I had with it is that it has an auto-donate feature where it will stop your miner for 15 minutes and it will mine into its own pools… which was a big problem for me because when I was mining maxcoin, I had to use a different miner application and I wasn’t using their service for mining, so it couldn’t “stop the server” so it ran two instances of the miner who froze the cards and I wasted 2 days without mining before I noticed… so if you want to disable the auto-donate, just do a ‘crontab -e’ as root and remove the scripts from crontab.
If you just want to use your desktop for mining, then you can use cgminer for AMD or cudaminer for Nvidia, but as you can see in the hardware comparison wiki, nvidia aren’t so good for mining. If you use cgminer, you must use version 3.7.2, because any version after that will *not* work for GPUs, as they dropped support for it. I use a fork of cgminer from “kalroth” (google kalroth cgminer) which has more option and bugfixes backported into 3.7.2. If you use scrypt-n, then you need vertminer (google it) which supports scrypt-n.
In theory it would take about 8 months to get back the price of a GPU from mining, but if the coin crashes, you lose the time you mined that coin, or if the coin gets a 10x increase in value, it could take only 1 month to get the money back.. you get the idea.
I’m not sure what other info I can give about the practical aspects of mining.. you get/have hardware, you get the miner, register in a pool, open the wallet, configure the pool in the miner (there are lots of instructions if you google for it) and start mining.. you put an auto-payout, and you hoard or sell depending on your preference.
For information, I’m using dogehouse.org as pool for dogecoin, and dedicatedpool.com for GRCE and SPA, and vertco.in for VTC.

Now for the last chapter of my book (lol). Mining BTC itself! Yes I know, I said earlier it’s impossible, but I actually said “impossible on a GPU”. I recently found this awesome site called cex.io in which you can trade BTC for GHS. It’s cloud mining but you don’t actually rent the GHs you buy them! Meaning that you get the GHS and you use it and mine from it forever or until you sell your GHS to someone else.. this means that you can buy 100GHS (would be around 600$ at current GHS price), let it mine for a while, which will give you about 6$ per day of revenue for 100GHS, then when you’re tired of it, you sell back those 100GHS. The issue here is two fold, the first is that the BTC network hashrate increases all the time, and the difficulty increases by about 30% per month I think, which means that the 6$ per day you get today, in one month will be much lower unless you buy more GHS.. The second issue is that since the value you can get from 1 GHS is lower every time the network hashrate increases, it means that the value, and so, the price of a GHS also decreases all the time, this means that when you sell back your GHS, you won’t get back the original amount of BTC (your 600$) you originally put in to buy them.. but hopefully, if you used it for long enough, then you will have earned those 600$ through mining already. I think it would take about 100 days to mine back your investment, then whatever you sell becomes extra money if you decide to sell your GHS.
The cool thing about cex.io is that you’re not renting the hardware, you’re buying it (or part of it), so if you have more GHS than what an ASIC miner provides, you can redeem the hardware and have it sent to you.. of course you’d have to pay electricty and maintain it yourself then, so not a great idea, but it’s still good to have that option and know that you actually do own the GHS that you bought. By the way, every earning you get from mining BTC, a portion of it gets taken to pay for electricity and maintenance, etc.. which is (according to them) about 13%, they have a complicated equation explained on the website, you pay per GHS you own and the time it’s been running (snice runs longer = more electricity use), this means that a portion of your earning goes into this maintenance fee, and the longer it takes to mine a BTC block, the higher the fee. Don’t forget, this is a pool so it’s a matter of luck.. I’ve seen it sometimes take a few seconds to find a block, or you find 10 blocks with 5 to 10 minutes between each block found, then you can spend 2 hours without finding any block.. the maximum fee I saw was about 50% and that was for the 2 hour delay before finding a block.. I guess overall it would average to 13% maybe.
If you want to give cex.io a try, I’d appreciate it if you use my referral link to create your account, because the referal gets 3% bonus GHS when a referred user trades GHS. So if you buy 10GHS, I’d get 0.3 extra (which I would lose if you sell, you can read the FAQ). Here is my referral link : https://cex.io/r/0/kakaroto/0/ please think about it when/if you decide to buy GHS to mine BTC directly.
The advantage of BTC is that its value is pretty stable.. sure, it fluctuates a lot, but it’s usually been between 550$ and 650$ in recent months, which is not like some altcoins (like maxcoin) which can see their value divided by 100 in a couple of weeks.
What I’ve done is use my GPU to mine altcoins, then I sold some of them (keeping AUR, VTC, GRCE, SPA and of course DOGE), and used that revenue to buy 11 GHS for myself and I’m leaving it mining now.. unfortunately, GHS price dropped in the past couple of months, so if I sell my 11GHS, then I’d have lost some money, but hopefully in a week or two, I’d be making a profit.
Another interesting point about cex.io is that you can ‘preorder’ GHS, they will add hardware on 26 of april and on 26 of may, and you can buy GHS for 0.008 for the april deployment and for 0.004 for the may deployment (currently active GHS can be traded for 0.011) so you could spend the money on the may GHS and triple the investment in 2 months… but on the other hand, it is possible that the GHS price drops by the 26th of may, and then you’d have bought GHS at the same price it would be selling on the day it becomes online, but you paid for it 2 months in advance instead of using those 2 months for mining with currently available GHS.. again, in this case, it’s a gamble. You can see the evolution of GHS value on cex.io for the past month to decide if it’s worth the risk.

Ok, I think that’s about it, I think I covered all the basics and advanced topics :-p

Don’t forget that if you decide to join cex.io, use my referral link here : https://cex.io/r/0/kakaroto/0/

Also, if you found this post interesting, you are more than welcome to send me tips to my BTC or DOGE wallets :

BTC : 15fXq5FzKaUArzQ8zBWfJADMn1qTQ5w5Y6

DOGE : DK4uNKX99VTgEv8BZdeepsYJHKhev3oR1j

Thanks for reading!

Tagged with:  

Eleganz release for Cobra ODE

On October 5, 2013, in Development, PS3, by kakaroto

Hi everyone,

It’s been a long time since I last blogged. Today I have some exciting news for you, as I have ported Eleganz, my homebrew manager, to the Cobra ODE.

A little while ago, I tweeted that if Cobra ever released their device and did provide an open source library for integration of other managers, I would port Eleganz to it, and today I am fulfilling that promise. I would like to thank the guys over at ps3crunch.net and ps3hax.net for testing this for me, particularly Abkarino, hyappon, freddy, magneto and Xodus69.

When I released Eleganz in November 2011, I left out one small thing on the TODO list, I wanted to see someone pick it up and add the code to exitspawn to actually make Eleganz execute the homebrew apps, but no one did that in almost a year now. I am a bit disappointed that the ps3 scene (homebrew devs, not users) didn’t pick it up, but it looked like no one was interested in maintaining Eleganz in my place. Today, I am happy to see that Eleganz is not throw-away code, as it can be useful to ODE users.

I can understand why Eleganz didn’t have much appeal in the world of CFW (it was originally intended to run on OFW if my HEN ever worked), but with the ODEs running on OFW, it’s perfect for the job. It’s simple, it’s beautiful and customizable!

Not only can Eleganz list the games from the Cobra ODE and allow you to select your iso, but it will also allow you to list and run homebrew apps that you can embed in the ISO file. This way you can get access to all your homebrew in a single place, without the need to restart the PS3 or boot the homebrew’s iso from the ODE. You can just extract the eleganz iso, and add homebrew apps (that are re-signed for running from a BD drive) to the iso’s PS3_GAME/USRDIR/HOMEBREW directory and recreate the iso with the cobra tool, and that’s it.

Note that this is not an indication of me getting back into the hacking scene. I have given up on the HEN long ago as I realized that there was no way (that I could find) to run homebrew on OFW, unless they are running from a disc. I may keep improving Eleganz in the near future, but I do not plan to do anything more than that for the ps3 scene at this point.

I would also like to tell everyone that there’s no need to worry, Eleganz will not become cobra-specific, as any feature I’d implement will benefit CFW as well as ODE users. I will be releasing an updated version for CFW users soon.

I’d also like to thank magneto and the Cobra team for offering to send me a Cobra ODE as a gift for porting Eleganz to it. Once I receive it, I plan on adding disc dumping capabilities to Eleganz and improve the user experience a little without relying on others to test it for me.

You can find the latest source code on github as always and compile it yourself or you can download the pre-compiled iso file from this link : http://www.multiupload.nl/GXBBI19VOL

I hope it gets used now and you all can enjoy it and I hope I can see some cool themes created for it now!

KaKaRoTo

Tagged with:  

libnice 0.1.4 released!

On February 22, 2013, in Development, libnice, by kakaroto

Hey everyone,

I have just released a new version of libnice, the NAT traversal library.

Version 0.1.4 has a few bug fixes but the major changes are the addition of an SDP parsing and generation API.

You can now more easily generate your credentials and candidates and parse them with a single API call, making it much easier to exchange candidates and establish a successful connection.

Also, I have added three examples to the examples/ subdirectoy from the libnice source tree. Those examples should help anyone learn how to use libnice and what to do in order to establish a successful connection.

The first example, simple-example.c will create a new agent, and gather its candidates and print out a single line to paste on the peer. It uses the signals to asynchronously wait for events and continue the code execution.

The second example, threaded-example.c, will run the mainloop on the main thread and do everything else sequentially in another thread, waiting for signals to release the libnice thread to continue processing.

The final example, sdp-example.c, is based on the threaded example but uses the new SDP generate/parsing API to generate the candidates and credentials line to exchange between the two instances. It will base64 the SDP to make it all fit into a single line, making it easier to exchange the SDP between clients without having to parse the multi-line SDP in the example, keeping it small and concise.

I hope you will find this release useful, let me know if you have any comments.

You can get the latest version here and the documentation has been updated here.

KaKaRoTo

Tagged with:  

Eleganz: The Elegant Homebrew Manager

On November 23, 2012, in Development, EFL, PS3, by kakaroto

Hi everyone,

Last year, in January, I decided to have some fun and write a homebrew application using the EFL libraries. I decided to work on a homebrew manager.. basically a replacement to the XMB. It went really well, and the development was really fast, and it was all thanks to the awesome API and capabilities of the EFL libraries. However, I became busy and was unable to continue… also, it was a bit slow and without proper hardware acceleration, it wouldn’t be as good as I hoped for, so I put the project on the side.
After many months, in September, thanks to gzorin’s work, we finally had a working and usable GL implementation and the EFL apps automatically gained from it by becoming hardware accelerated. My homebrew manager was much better! but I still needed to finish a few things and I didn’t have time so I put to rest again.

Today, I have decided to release this homebrew application, *as is* for everyone’s enjoyment! This means that it is not fully working, it might still have some bugs here and there, but it is still a homebrew app that people can use and have some fun with. Most importantly it will serve 4 purposes :

  • Maybe re-awaken  this dying PS3 homebrew scene
  • Be a good “exercise to the community” for finishing it up
  • Be a good example of what can be done with the EFL
  • Bring non-developers into writing EFL themes for the app

 

I introduce to you, Eleganz! The Elegant Homebrew Manager! A little homebrew app that lets you install pkg files and run your games directly from it. Here is the mandatory screencast video :


YouTube Link toEleganz screencast

 

I have published my app in both github and on ps3dev’s gitorious. and you can also download a pre-compiled .pkg for your PS3 to have fun with it.

Here are some highglights of the application (features, limitations and bugs) :

  • The whole User Interface is completely customizable with themes
  • Installs .pkg files locally to its own data directory (won’t be visible in the real XMB, unless someone reverses the database format)
  • Does not yet run games (it’s for you to do it, use ps3load as reference maybe…)
  • Current theme is missing proper theme/images for the progressbar windows (default exquisite/E17 theme used)
  • System freezes for a few milliseconds when it tries to load a game’s background image (might be fixed if we implement a pthread library and threading support in the EFL)
  • Apparently crashes when it exits (bug)

The homebrew app comes with two themes, a dark and light theme. I like the dark one so I chose that as the default (oh, ignore that grey background ‘default’ one from that screencast video, that was just for testing). I wrote the user interface for the theme (the Edje files) while opium designed all the graphics. The theme engine in the EFL is extremely powerful, so I hope I will see tons of themes popping up. And I do not mean “change the images” themes, I want real themes, where the whole UI is different, a vertical XMB, a circular one, a 3D theme with perspective/depth for the icons, a dynamic/moving background, etc… You can learn about the .edj/.edc file format here and don’t forget to check the EDC reference wiki.

I hope to see the community pick this up and have fun with it!

That’s about it, enjoy it, and send me your patches! I’ll be waiting :)

KaKaRoTo

 

p.s: Forgot to say that the rules/naming conventions/etc.. of the EDC files are explained here. If a .edj file doesn’t have the appropriate parts/groups, then it will be ignored and will not show on the UI.

p.p.s: You can install the EFL on windows and have access to edje_cc to compile your .edc into .edj.

p.p.p.s: Damn, I  keep forgetting stuff.. by the way, the whole Eleganz application works just fine on the PC too, I did all my development on the PC (that screencast was actually on Linux), *then* I tried it on the PS3 and it just worked.. so for theme development, it should be pretty easy to test without the need of a PS3.

Tagged with:  

UVC H264 Encoding cameras support in GStreamer

On September 21, 2012, in Development, by kakaroto

More and more people are doing video conferencing everyday, and for that to be possible, the video has to be encoded before being sent over the network. The most efficient and most popular codec at this time is H264, and since the UVC (USB Video Class) specification 1.1, there is support for H264 encoding cameras.

One such camera is the Logitech C920. A really great camera which can produce a 1080p H264-encoded stream at 30 fps.  As part of my job for Collabora, I was tasked to add support for such a camera in GStreamer. After months of work, it’s finally done and has now been integrated upstream into gst-plugins-bad 0.10 (port to GST 1.0 pending).

One important aspect here is that when you are capturing a high quality, high resolution H264 stream, you don’t want to be wasting your CPU to decode your own video in order to show a preview window during a video chat, so it was important for me to be able to capture both H264 and raw data from the camera. For this reason, I have decided to create a camerabin2-style source: uvch264_src.

Uvch264_src is a source which implements the GstBaseCameraSrc API. This means that instead of the traditional ‘src’ pad, it provides instead three distinct source pads: vidsrc, imgsrc and vfsrc. The GstBaseCameraSrc API is based heavily on the concept of a “Camera” application for phones. As such, the vidsrc is meant as a source for recording video, imgsrc as a source for taking a single-picture and vfsrc as a source for the viewfinder (preview) window. A ‘mode’ property is used to switch between video-mode and image-mode for capture. The uvch264_src source only supports video mode however, and the imgsrc will never be used.

When the element goes to PLAYING, only the vfsrc will output data, and you have to send the “start-capture” action signal to activate the vidsrc/imgsrc pads, and send the “stop-capture” action signal to stop capturing from the vidsrc/imgsrc pads. Note that vfsrc will be outputting data when in PLAYING, so it must always be linked (link it to fakesink if you don’t need the preview, otherwise you’ll get a not-linked error). If you want to test this on the command line (gst-launch) you can set the ‘auto-start’ property to TRUE and the uvch264_src will automatically start the capture after going to PLAYING.

You can request H264, MJPG, and raw data from the vidsrc, but only MJPG and raw data from the vfsrc. When requesting H264 from the vidsrc, then the max resolution for the vfsrc will be 640×480, which can be served as jpg or as raw (decoded from jpg). So if you don’t want to use any CPU for decoding, you should ask for a raw resolution lower than 432×240 (with the C920) which will directly capture YUV from the camera. Any higher resolution won’t be able to go through the usb’s bandwidth and the preview will have to be captured in mjpg (uvch264_src will take care of that for you).

The source has two types of controls, the static controls which must be set in READY state, and DYNAMIC controls which can be dynamically changed in PLAYING state. The description of each property will specify whether that property is a static or dynamic control, as well as its flags. Here are the supported static controls : initial-bitrate, slice-units, slice-mode, iframe-period, usage-type, entropy, enable-sei, num-reorder-frames, preview-flipped and leaky-bucket-size. The dynamic controls are : rate-control,  fixed-framerate, level-idc, peak-bitrate, average-bitrate, min-iframe-qp,  max-iframe-qp, min-pframe-qp,  max-pframe-qp, min-bframe-qp,  max-bframe-qp, ltr-buffer-size and ltr-encoder-control.

Each control will have a minimum, maximum and default value, and those are specific to each camera and need to be probed when the element is in READY state. For that reason, I have added three element actions to the source in order to probe those settings : get-enum-setting, get-boolean-setting and get-int-setting.
These functions will return TRUE if the property is valid and the information was successfully retrieved, or FALSE if the property is invalid (giving an
invalid name or a boolean property to get_int_setting for example) or if the camera returned an error trying to probe its capabilities.
The prototype of the signals are :

  • gboolean get_enum_setting (GstElement *object, char *property, gint *mask, gint *default);
    Where the mask is a bit field specifying which enums can be set, where the bit being set is (1 << enum_value).
    For example, the ‘slice-mode’ enum can only be ignored (0) or slices/frame (3), so the mask returned would be : 0×09
    That is equivalent to (1 << 0 | 1 << 3) which is :
    (1 << UVC_H264_SLICEMODE_IGNORED) | (1 << UVC_H264_SLICEMODE_SLICEPERFRAME)
  • gboolean get_int_setting (GstElement *object, char *property, gint *min, gint *def, gint *max);
    This one gives the minimum, maximum and default values for a property. If a property has min and max to the same value, then the property cannot be changed, for example the C920 has num-reorder-frames setting: min=0, def=0 and max=0, and it means the C920 doesn’t support reorder frames.
  • gboolean get_boolean_setting (GstElement *object, char *property, gboolean *changeable, gboolean *default_value);
    The boolean value will have changeable=TRUE only if changing the property will have an effect on the encoder, for example, the C920 does not support the preview-flipped property, so that one would have changeable=FALSE (and default value is FALSE in this case), but it does support the enable-sei property so that one would have changeable=TRUE (with a default value of FALSE).

This should give you all the information you need to know which settings are available on the hardware, and from there, be able to control the properties
that are available.

Since these are element actions, they are called this way :

  1. gboolean return_value, changeable, default_bool;
  2. gint mask, minimum, maximum, default_int, default_enum;
  3.  
  4. g_signal_emit_by_name (G_OBJECT(element), "get-enum-setting", "slice-mode", &mask, &default_enum, &return_value, NULL);
  5. g_signal_emit_by_name (G_OBJECT(element), "get-boolean-setting", "enable-sei", &changeable, &default_bool, &return_value, NULL);
  6. g_signal_emit_by_name (G_OBJECT(element), "get-int-setting", "iframe-period", &minimum, &default_int &maximum, &return_value, NULL);

Apart from that, the source also supports the GstForceKeyUnit video event for dynamically requesting keyframes, as well as custom-upstream events to control LTR (Long-Term Reference frames), bitrate, QP, rate-control and level-idc, through, respectively, the uvc-h264-ltr-picture-control, uvc-h264-bitrate-control, uvc-h264-qp-control, uvc-h264-rate-control and uvc-h264-level-idc custom upstream events (read the code for more information!). The source also supports receiving the ‘renegotiate’ custom upstream event which will make it renegotiate the according to the caps on its pads. This is useful if you want to enable/disable h264 streaming or switch resolution/framerate easily while the pipeline is running; Simply change your caps and send the renegotiate event.

I have written a GUI test application which you can use to test the camera and the source’s various features. It can also serve as a reference implementation on how the source can be used. The test application resides in gst-plugins-bad, under tests/examples/uvch264/ (make sure to run it from its source directory though).

 

uvch264_src test application (click to enlarge)

You can also use this example gst-launch pipeline for testing the capture of the camera. This will capture a small preview window as well as an h264 stream in 1080p that will be decoded locally :

gst-launch uvch264_src device=/dev/video1 name=src auto-start=true src.vfsrc ! queue ! “video/x-raw-yuv,width=320,height=240,framerate=30/1″ ! xvimagesink src.vidsrc ! queue ! video/x-h264,width=1920,height=1080,framerate=30/1,profile=constrained-baseline ! h264parse ! ffdec_h264 ! xvimagesink

That’s about it. If you are interested in using uvch264_src to capture from one of the UVC H264 encoding cameras, make sure you upgrade to the latest git versions of gstreamer, gst-plugins-base, gst-plugins-good and gst-plugins-bad (or 0.10.37+ for gstreamer/gst-plugins-base, 0.10.32 for gst-plugins-good and 0.10.24 for gst-plugins-bad, whenever those versions get released).

I would like to thank Collabora and Cisco for sponsoring me to work on this great project, it couldn’t have been possible without them!

If you have any more questions about this subject, feel free to contact me.

Enjoy!

 

Tagged with:  

RSXGL working and usable

On September 11, 2012, in Development, EFL, PS3, by kakaroto

Hi everyone!

When the PS3 homebrew scene started, a lot of people were complaining that it wasn’t possible to write 3D games for the PS3 because of its lack of OpenGL library.
Almost a year ago, Alex Betts thought he would tackle this problem and he started working on RSXGL… an implementation of the OpenGL 3.1 specification written from scratch targeting the PS3′s RSX. Anyone in their right mind would say that it’s impossible, that it’s too much work, but Alex spent the last year working on it, alone, until it became usable. You can read some news about it here.
For some reason though, no one used it to build their own apps. Maybe the status of the project was scaring them, it was said to be incomplete, there was no GLSL support, etc…
I am writing today to tell you that RSXGL is perfectly usable! It supports online GLSL compilation, as well as any feature you might want. As proof, I have written a new hardware accelerated engine for the EFL using RSXGL and it worked great! Alex and I spent a lot of time testing and fixing all the issues that were in RSXGL that were made visible by the EFL’s GL engine and I am happy to say that it’s working now. Expedite is able to run most of its tests at 50 to 60 fps on 1080p resolution (instead of the average of 5 to 10 fps it had on 720p before).
You can see performance tests right here (Running some tests from expedite) :
Software rendering: http://dl.dropbox.com/u/22642664/expedite_psl1ght.log
RSXGL rendering: http://dl.dropbox.com/u/22642664/expedite_rsxgl.log

Please give RSXGL a try. Also, you can get the latest EFL version from my repository, which includes the gl engine for ps3. Now, any EFL application will be automatically hardware accelerated thanks to RSXGL. I hope we can see some new games (or old GL games being ported) soon!

RSXGL : https://github.com/gzorin/RSXGL
EFL : https://github.com/kakaroto/e17

Enjoy!

 

Exquisite tool becomes a library!

On July 21, 2012, in Development, EFL, PS3, by kakaroto

The exquisite tool that comes with Enlightenment is a nice, pure edje application that is used for showing boot process splash screens. I thought it was a nice splash screen and more generally, a nice progress bar and wanted to use it in my own apps.

I have modified the exquisite tool to become a library so it can be used by others in their applications, while keeping the exquisite and exquisite-writer tools intact (they will now depend on libexquisite.so though).

Since it’s a very simple feature (only a couple hundred lines of code), the API is simple as well. Here’s an example of use :

  1.   Evas_Object *obj = exquisite_object_add (evas, theme);
  2.   evas_object_show (obj);
  3.  
  4.   exquisite_object_title_set (obj, "Title of the screen");
  5.   exquisite_object_message_set (obj, "My Message");
  6.  
  7.   int test_id = exquisite_object_text_add (obj, "First test");
  8.   int second_test = exquisite_object_text_add (obj, "Second text!");
  9.   exquisite_object_status_set (obj, test_id, "FAIL", EXQUISITE_STATUS_TYPE_FAILURE);
  10.   exquisite_object_status_set (obj, second_test, "OK", EXQUISITE_STATUS_TYPE_SUCCESS);
  11.  
  12.   exquisite_object_pulsate (obj);
  13.   exquisite_object_progress_set (obj, 0.95);

That’s it, and you get a nice screen with title, message, text area for status messages (if you want it), and a progress bar. This also means you can use the default theme from exquisite or write your own using the same theme specification which can later on be used by others.

Please review the API provided, and this is the right time to suggest any changes to the API or to improve on the library, so let us know what you think!

Tagged with:  

A couple of months ago I bought a new laptop with 8GB of RAM, but I realized I was running on a 32 bits system which meant I couldn’t use all my RAM. I had to switch to 64 bits. It takes so much time for me to restore my system that I didn’t have the courage to go through it again (did it last year, switched from Debian to Fedora, took me a week), so I stayed with 32 bits. Yesterday I had to upgrade to Fedora 16 and decided to do the switch to 64 bits at the same time… I’d like to share my experience with you!

First of all, I had to download the 64 bits version of the fedora CD which is not the default download on the website, I had to click on the small “more download options” to get the choice and I realized that’s how I got the 32 bit  install in the first place (Fedora download page should definitely list both links). Then I made a backup of all the installed packages on my system so I can restore them on the new system :

 yum -C info $(rpm -qa) | grep “Name   :” | cut -c 15- > packages-list.log

This will list all of the packages installed, and ask yum for the exact name of the package (instead of “git-1.7.6.5-1.fc15.i686″, it becomes “git”).. if you have a better method of doing that, let me know, but this did the trick for me.

Update: A better method was given to me by Hansen and Richard Godbee in the comments : rpm -qa –qf “%{name}\n” > packages-list.log

I obviously had a separate partition for the  /home directory, which made things easier, so I backed up in it the important directories which were: /opt, /root, /etc, /usr/local and my scratchbox home dir. Then the moment of truth, reboot into the live cd, install it, make sure not to format the /home partition, and reboot into the new 64 bits system.

First of all, as soon as I tried to login, gnome 3 would completely crash and would not let me log in, so I had to create a new user, login into gnome 3, then “ls -la” the files in the new user’s home dir, then delete (move away) those same files/directories from my own home dir, so that gnome doens’t crash anymore… apparently, my settings suddenly became incompatible or something… It’s important to note that I had some further problems later and I had to copy back .gnome2/keyrings otherwise the gnome-keyring daemon would freeze.

To restore all the packages that I had before, I first had to re-install (manually) the rpmfusion repository (free and nonfree), then I just did a simple :

yum install $(cat packages-list.log)

And after 1.2GB of downloads and 1020 package installs, my system was technically “restored” to how it was before the format. I look at the “No package foobar” lines given by yum at that point which told me what I needed to install manually (opera, skype, dropbox), which I did, and a few libs that apparently don’t exist anymore in Fedora 16. Now I just had to restore the /opt for some apps I had in there (and recompile the EFL/E17),  copy the Enlightenment.desktop file to /usr/share/xsessions, restore my /etc/hosts (which had some custom entries), restore some custom scripts I wrote into /usr/local/bin and recompile the libraries I was working on and had installed in /usr/local (gstreamer, libnice, farstream). I also had to install a few 32 bit libraries so I could install skype (which only comes in 32 bit flavor).

It took me about a day of work/compilation, but now I feel back home, don’t notice any difference in my system other than the fact that I will now be writing 32-bits bugs instead of 64-bits bugs :)

 

 

Tagged with:  

How the ECDSA algorithm works

On January 31, 2012, in Development, PS3, by kakaroto

To popular demand, I have decided to try and explain how the ECDSA algorithm works. I’ve been struggling a bit to understand it properly and while I found a lot of documentation about it, I haven’t really found any “ECDSA for newbies” anywhere. So I thought it would be good to explain in simple terms how it works so others can learn from my research. I have found some websites that explain the basic principles but nowhere near enough to actually understand it, others that explains things without any basics, making it incomprehensible, and others that go way too deep into the the mathematics behind it.

ECDSA stands for “Elliptic Curve Digital Signature Algorithm”, it’s used to create a digital signature of data (a file for example) in order to allow you to verify its authenticity without compromising its security. Think of it like a real signature, you can recognize someone’s signature, but you can’t forge it without others knowing. The ECDSA algorithm is basically all about mathematics.. so I think it’s important to start by saying : “hey kids, don’t slack off at school, listen to your teachers, that stuff might be useful for you some day!” :) But these maths are fairly complicated, so while I’ll try to vulgarize it and make it understandable for non technical people, you will still probably need some knowledge in mathematics to understand it properly. I will do this in two parts, one that is a sort of high level explanation about how it works, and another where I dig deeper into its inner workings to complete your understanding. Note however that I’ve just recently learned this stuff, so I’m definitely not an expert on the matter.

So the principle is simple, you have a mathematical equation which draws a curve on a graph, and you choose a random point on that curve and consider that your point of origin. Then you generate a random number, this is your private key, you do some magical mathematical equation using that random number and that “point of origin” and you get a second point on the curve, that’s your public key. When you want to sign a file, you will use this private key (the random number) with a hash of the file (a unique number to represent the file) into a magical equation and that will give you your signature. The signature itself is divided into two parts, called R and S. In order to verify that the signature is correct, you only need the public key (that point on the curve that was generated using the private key) and you put that into another magical equation with one part of the signature (S), and if it was signed correctly using the the private key, it will give you the other part of the signature (R). So to make it short, a signature consists of two numbers, R and S, and you use a private key to generate R and S, and if a mathematical equation using the public key and S gives you R, then the signature is valid. There is no way to know the private key or to create a signature using only the public key.

Alright, now for the more in depth understanding, I suggest you take an aspirin right now as this might hurt! :P

Let’s start with the basics (which may be boring for people who know about it, but is mandatory for those who don’t) : ECDSA uses only integer mathematics, there are no floating points (this means possible values are 1, 2, 3, etc.. but not 1.5..),  also, the range of the numbers is bound by how many bits are used in the signature (more bits means higher numbers, means more security as it becomes harder to ‘guess’ the critical numbers used in the equation), as you should know, computers use ‘bits’ to represent data, a bit is a ‘digit’ in binary notation (0 and 1) and 8 bits represent one byte. Every time you add one bit, the maximum number that can be represented doubles, with 4 bits you can represent values 0 to 15 (for a total of 16 possible values), with 5 bits, you can represent 32 values, with 6 bits, you can represent 64 values, etc.. one byte (8 bits) can represent 256 values, and 32 bits can represent 4294967296 values (4 Giga).. Usually ECDSA will use 160 bits total, so that makes… well, a very huge number with 49 digits in it…

ECDSA is used with a SHA1 cryptographic hash of the message to sign (the file). A hash is simply another mathematical equation that you apply on every byte of data which will give you a number that is unique to your data. Like for example, the sum of the values of all bytes may be considered a very dumb hash function. So if anything changes in the message (the file) then the hash will be completely different. In the case of the SHA1 hash algorithm, it will always be 20 bytes (160 bits). It’s very useful to validate that a file has not been modified or corrupted, you get the 20 bytes hash for a file of any size, and you can easily recalculate that hash to make sure it matches. What ECDSA signs is actually that hash, so if the data changes, the hash changes, and the signature isn’t valid anymore.

Now, how does it work? Well Elliptic Curve cryptography is based on an equation of the form :

y^2 = (x^3 + a * x + b) mod p

First thing you notice is that there is a modulo and that the ‘y‘ is a square. This means that for any x coordinate, you will have two values of y and that the curve is symmetric on the X axis. The modulo is a prime number and makes sure that all the values are within our range of 160 bits and it allows the use of “modular square root” and “modular multiplicative inverse” mathematics which make calculating stuff easier (I think). Since we have a modulo (p) , it means that the possible values of y^2 are between  0 and p-1, which gives us p total possible values. However, since we are dealing with integers, only a smaller subset of those values will be a “perfect square” (the square value of two integers), which gives us N possible points on the curve where N < p (N being the number of perfect squares between 0 and p). Since each x will yield two points (positive and negative values of the square-root of y^2), this means that there are N/2 possible ‘x‘ coordinates that are valid and that give a point on the curve. So this elliptic curve has a finite number of points on it, and it’s all because of the integer calculations and the modulus. Another thing you need to know about Elliptic curves, is the notion of “point addition“. It is defined as adding one point P to another point Q will lead to a point S such that if you draw a line from P to Q, it will intersect the curve on a third point R which is the negative value of S (remember that the curve is symmetric on the X axis). In this case, we define R = -S to represent the symmetrical point of R on the X axis. This is easier to illustrate with an image : So you can see a curve of the form y^2 = x^3 + ax + b (where a = -4 and b = 0), which is symmetric on the X axis, and where P+Q is the symmetrical point through X of the point R which is the third intersection of a line going from P to Q. In the same manner, if you do P + P,  it will be the symmetrical point of R which is the intersection of the line that is a tangent to the point P.. And P + P + P is the addition between the resulting point of P+P with the point P since P + P + P can be written as (P+P) + P.. This defines the “point multiplication” where k*P is the addition of the point P to itself k times… here are two examples showing this :  

Here, you can see two elliptic curves, and a point P from which you draw the tangent, it intersects the curve with a third point, and its symmetric point it 2P, then from there, you draw a line from 2P and P and it will intersect the curve, and the symmetrical point is 3P. etc… you can keep doing that for the point multiplication. You can also already guess why you need to take the symmetric point of R when doing the addition, otherwise, multiple additions of the same point will always give the same line and the same three intersections.

One particularity of this point multiplication is that if you have a point R = k*P, where you know R and you know P, there is no way to find out what the value of ‘k‘ is. Since there is no point subtraction or point division, you cannot just resolve k = R/P. Also, since you could be doing millions of  point additions, you will just end up on another point on the curve, and you’d have no way of knowing “how” you got there. You can’t reverse this operation, and you can’t find the value ‘k‘ which was multiplied with your point P to give you the resulting point R.

This thing where you can’t find the multiplicand even when you know the original and destination points is the whole basis of the security behind the ECDSA algorithm, and the principle is called a “trap door function“.

Now that we’ve handled the “basics”, let’s talk about the actual ECDSA signature algorithm. For ECDSA, you first need to know your curve parameters, those are a, b, p, N and G. You already know that ‘a‘ and ‘b‘ are the parameters of the curve function (y^2 = x^3 + ax + b), that ‘p‘ is the prime modulus,  and that ‘N‘ is the number of points of the curve, but there is also ‘G‘ that is needed for ECDSA, and it represents a ‘reference point’ or a point of origin if you prefer. Those curve parameters are important and without knowing them, you obviously can’t sign or verify a signature. Yes, verifying a signature isn’t just about knowing the public key, you also need to know the curve parameters for which this public key is derived from.

So first of all, you will have a private and a public key.. the private key is a random number (of 20 bytes) that is generated, and the public key is a point on the curve generated from the point multiplication of G with the private key. We set ‘dA‘ as the private key (random number) and ‘Qa‘ as the public key (a point), so we have : Qa = dA * G (where G is the point of reference in the curve parameters).

So how do you sign a file/message ? First, you need to know that the signature is 40 bytes and is represented by two values of 20 bytes each, the first one is called R and the second one is called S.. so the pair (R, S) together is your ECDSA signature.. now here’s how you can create those two values in order to sign a file.. first you must generate a random value ‘k‘ (of 20 byes), and use point multiplication to calculate the point P=k*G. That point’s x value will represent ‘R‘. Since the point on the curve P is represented by its (x, y) coordinates (each being 20 bytes long), you only need the ‘x‘ value (20 bytes) for the signature, and that value will be called ‘R‘. Now all you need is the ‘S‘ value.

To calculate S, you must make a SHA1 hash of the message, this gives you a 20 bytes value that you will consider as a very huge integer number and we’ll call it ‘z‘. Now you can calculate S using the equation :

S = k^-1 (z + dA * R) mod p

Note here the k^-1 which is the ‘modular multiplicative inverse‘ of k… it’s basically the inverse of k, but since we are dealing with integer numbers, then that’s not possible, so it’s a number such that (k^-1 * k ) mod p is equal to 1. And again, I remind you that k is the random number used to generate R, z is the hash of the message to sign, dA is the private key and R is the x coordinate of k*G (where G is the point of origin of the curve parameters).

Now that you have your signature, you want to verify it, it’s also quite simple, and you only need the public key (and curve parameters of course) to do that. You use this equation to calculate a point P :

P=  S^-1*z*G + S^-1 * R * Qa

If the x coordinate of the point P is equal to R, that means that the signature is valid, otherwise it’s not.

Pretty simple, huh? now let’s see why and how… and this is going to require some mathematics to verify :

We have :

P = S^-1*z*G + S^-1 * R *Qa

but Qa = dA*G, so:

P = S^-1*z*G + S^-1 * R * dA*G = S^-1 (z + dA* R) * G

But the x coordinate of P must match R and R is the x coordinate of k * G, which means that :

k*G = S^-1 (z + dA * R) *G

we can simplify by removing G which gives us :

k = S^-1(z + dA * R)

by inverting k and S, we get :

S = k^-1 (z + dA *R)

and that is the equation used to generate the signature.. so it matches, and that is the reason why you can verify the signature with it.

You can note that you need both ‘k‘ (random number) and ‘dA‘ (the private key) in order to calculate S, but you only need R and Qa (public key) to validate the signature. And since R=k*G and Qa = dA*G and because of the trap door function in the ECDSA point multiplication (explained above), we cannot calculate dA or k from knowing Qa and R, this makes the ECDSA algorithm secure, there is no way of finding the private keys, and there is no way of faking a signature without knowing the private key.

The ECDSA algorithm is used everywhere and has not been cracked and it is a vital part of most of today’s security.

Now I’ll discuss on how and why the ECDSA signatures that Sony  used in the PS3 were faulty and how it allowed us to gain access to their private key.

So you remember the equations needed to generate a signature.. R = k*G and S= k^-1(z + dA*R) mod p.. well this equation’s strength is in the fact that you have one equation with two unknowns (k and dA) so there is no way to determine either one of those. However, the security of the algorithm is based on its implementation and it’s important to make sure that ‘k‘ is randomly generated and that there is no way that someone can guess, calculate, or use a timing attack or any other type of attack in order to find the random value ‘k‘. But Sony made a huge mistake in their implementation, they used the same value for ‘k‘ everywhere, which means that if you have two signatures, both with the same k, then they will both have the same R value, and it means that you can calculate k using two S signatures of two files with hashes z and z’ and signatures S and S’ respectively :

S – S’ = k^-1 (z + dA*R) – k^-1 (z’ + da*R) = k^-1 (z + da*R – z’ -dA*R) = k^-1 (z – z’)

So : k = (z – z’) / (S – S’)

Once you know k, then the equation  for S because one equation with one unknown and is then easily resolved for dA :

dA = (S*k – z) / R

Once you know the private key dA, you can now sign your files and the PS3 will recognize it as an authentic file signed by Sony. This is why it’s important to make sure that the random number used for generating the signature is actually “cryptographically random”.  This is also the reason why it is impossible to have a custom firmware above 3.56, simply because since the 3.56 version, Sony have fixed their ECDSA algorithm implementation and used new keys for which it is impossible to find the private key.. if there was a way to find that key, then the security of every computer, website, system may be compromised since a lot of systems are relying on ECDSA for their security, and it is impossible to crack.

Finally! I hope this makes the whole algorithm clearer to many of you.. I know that this is still very complicated and hard to understand. I usually try to make things easy to understand for non technical people, but this algorithm is too complex to be able to explain in any simpler terms. After all that’s why I prefer to call it the MFET algorithm (Mathematics For Extra Terrestrials) :)

But if you are a developer or a mathematician or someone interested in learning about this because you want to help or simple gain knowledge, then I’m sure that this contains enough information for you to get started or to at least understand the concept behind this unknown beast called “ECDSA”.

That being said, I’d like to thank a few people who helped me understand all of this, one particularly who wishes to remain anonymous, as well as the many wikipedia pages I linked to throughout this article, and Avi Kak thanks to his paper explaining the mathematics behind ECDSA, and from which I have taken those graph images aboves.

P.s: In this article, I used ’20 bytes’ in my text to talk about the ECDSA signature because that’s what is usually used as it matches the SHA1 hash size of 20 bytes and that’s what the PS3 security uses, but the algorithm itself can be used with any size of numbers. There may be other inaccuracies in this article, but like I said, I’m not an expert, I just barely learned all of this in the past week.

Tagged with:  

Status update on the PS3 4.0 HEN

On January 19, 2012, in Development, PS3, by kakaroto

Here’s a “quick” status update on the 4.00 HEN (Homebrew ENabler) for PS3.

Following my clarifications from almost 2 months ago here, there has been a lot of progress. We have not been slacking off, we’re a group of about 10 developers working together for the last 2 months, for sometimes 15 hours everyday in order to bring back homebrew support to the latest version of the PS3.

There are three major parts to the HEN, first, getting the packages to install on the PS3, that part is done, completed, tested, debugged, etc.. the second part is to get the apps to run, that one still has major issues… the last part is something I will not discuss for now (it’s a surprise) but it’s about 60% to 70% done (and it has nothing to do with peek&poke and has nothing to do with backup managers or anything like that. This is and will stay a piracy-free solution for the PS3).

Now, running apps is the biggest challenge that we’ve been working on for the past 2 months. As some of you know, if you’ve been following me on Twitter, we originally had hoped for Mathieulh to give us the “npdrm hash algorithm” that was necessary to run the apps, but he was reluctant, he kept doing his usual whore so people would kiss his feet (or something else) so he’d feel good about himself. But in the end, he said that he refuses to give us the needed “npdrm hash algorithm” to make it work… So what I initially thought would be “this will be released next week” ended up taking a lot more time than expected, and we’re still nowhere near ready to make it work.

Mathieulh kept tossing his usual “riddles” which he thinks are “very helpful for those who have a brain”, and which pisses off anyone who actually does… so he told us that the solution to all our problems was to look in appldr of the 3.56 firmware.. and that it was something lv1 was sending appldr which made the “hash check” verified or not… so we spent one month and a lot of sweat and after killing a few of our brain cells out of exhaustion, we finally concluded that it was all bullshit. After one month of reading assembly code and checking and double-checking our results, we finally were able to confirm that that hash algorithm was NOT in the 3.56 firmware like he told us (at all).

He said  that it was an AES OMAC hash, but after tracking all the uses of the OMAC functions in appldr, we found that it was not used for the “hash”…  he then said “oh, I meant HMAC“, so we do that again and again come up with the same conclusion, then we’re sure it’s not in appldr, and then he says “ah no, it’s in lv1“.. have a look for yourself to what he decided to write : http://www.ps3devwiki.com/index.php?title=Talk:KaKaRoTo_Kind_of_%C2%B4Jailbreak%C2%B4

That happened after the huge twitter fight I had with him for being his usual arrogant ass and claiming that he “shared” something (For your information, the code that he shared was not his own, I have proof of that too (can’t show you the proof because even if I don’t respect him, I gave him my word to not share what he gave me, and I respect my word) since he forgot to remove the name of the original developer from one of the files… also it was completely useless and was not used at all, just made me waste a day reading the crappy undocumented code. So why is he still trying to force his “advice” through these riddles even after we had that fight? Well to sabotage us and make us lose all those months of hard work!

So anyways, we had all accepted that Mathieulh was full of shit (we knew before, but we gave him the benefit of the doubt) and decided to continue working without considering any of his useless riddles. So we then tried to exploit/decrypt the 3.60+ firmware in order to get the algorithm from there.

Now, a few more weeks later, we finally have succeeded in fully understanding that missing piece from the “npdrm hash algorithm”,  and here it is for everyone’s pleasure with some prerequisite explanation :

A game on the PS3 is an executable file in a format called a “SELF“file (kind of like .exe on windows), those “self” files are cryptographically signed and encrypted.. For PSN games (games that do not run from a bluray disc), they need to have an additional security layer called “NPDRM”. So a “npdrm self” is basically an executable that is encrypted and signed, then re-encrypetd again with some additional information. On 3.55 and lower, we were able to encrypt and sign our own self files so they would look like original (made by sony) “npdrm self” files, and the PS3 would run them without problem. However, it wasn’t really like an original file.. a real NPDRM self file had some additional information that the PS3 simply ignored, it did not check for that information, so we could put anything in it, and it worked. Since the 3.60 version, the PS3 now also validates this additional information, so it can now differentiate between NPDRM self files created by sony and the ones that we create ourselves for homebrew. That’s the “npdrm hash algorithm” that we have been trying to figure out, because once we can duplicate that information in the proper manner, then the PS3 will again think that those files are authentic and will let us play them.

Another important point to explain, I said a few times that the files are “signed”.. this means that there is an “ECDSA signature” in the file which the PS3 can verify. The ECDSA signature is something that allows the PS3 to verify if the file has been modified or not.. it is easy to validate the signature, but impossible to create one without having access to the “private keys” (think of it like a real signature, you can see your dad’s signature and recognize it, but you can’t sign it exactly like him, and you can recognize if your brother tried to forge his signature). So how were we able to sign the self files that were properly authenticated on 3.55? That’s because this “ECDSA signature” is just a very complicated mathematical equation (my head still hurts trying to fully understand it, but I might blog about it in the future and try to explain it in simple terms if people are interested you can learn about it here), and one very important part of this mathematical equation is that you need to use a random number to generate the signature, but Sony had failed and used the same number every time.. by doing that, it was easy to just find the private key (which allows us to forge perfectly the signature) by doing some mathematical equation on it. So to summarize, a “signed file” is a file which is digitally signed with an “ECDSA signature” that cannot be forged, unless you have the “private key” for it, which is impossible to obtain usually, but we were able to obtain it because Sony failed in implementing it properly.

Now, back on topic.. so what is this missing “npdrm hash algorithm” that we need? well it turns out that the “npdrm self” has a second signature, so it’s a “encrypted and signed self file” with an additional layer of security (the NPDRM layer) which re-encrypts it and re-signs it again. That second signature was not verified in 3.55 and is now verified since the 3.60 version of the PS3 firmware.

One important thing to note is that Sony did NOT make the same mistake with this signature, they always used a random number, so it it technically impossible to figure out the private key for it. To be more exact, this is the exact same case as the .pkg packages you install on the PS3, you need to patch the firmware (making it cfw) so that those .pkg files can be installed, and that’s because the .pkg files are signed with an ECDSA signature for which no one was able to get the private key. That’s why we call them “pseudo-retail packages” or “unsigned packages”.

The signature on the NPDRM self file uses the exact same ECDSA curve and the same key as the one used in PS3 .pkg files, so no one has (or could have) the private key for it. What this means is that, even though we finally figured out the missing piece and we now know how the NPDRM self is built, we simply cannot duplicate it.

The reason we wasted 2 months on this is because Mathieulh lied by saying that he can do it.. remember when the 4.0 was out and I said “I can confirm that my method still works” then he also confirmed that his “npdrm hash algorithm” still works too? well he didn’t do anything to confirm, he just lied about it because there is no way that he could have verified it because he doesn’t have the private key.

I said I will provide proof of the lies that Mathieulh gave us, so here they are : he said it’s in 3.56, that was a lie, he said it’s an AES OMAC, that was a lie,  he said it’s an HMAC, that was a lie, he said it’s in appldr, that was a lie, he said it’s in lv1, that was a lie, he said that he can do it, that was a lie, he said that “it takes one hour to figure it out if you have a brain”, that was a lie, he said that he verified it to work on 4.0, that was a lie, he said that he had the algorithm/keys, that was a lie, he said that once we know the algorithm used, we can reproduce it, that was a lie, he kept referring to it as “the hash”, that was wrong. The proof ? It’s an ECDSA signature, it’s not a hash (two very different terms for different things), it was verified by vsh.self, it was not in lv2, or lv1, or appldr, and the private key is unaccessible, so there is no way he could build his own npdrm self files. Now you know the real reason why he refused to “share” what he had.. it’s because he didn’t have it…

So why do all this? was it because his arrogance didn’t allow him to admit not knowing something? or was it because he wanted to make us lose all this time? To me, it looks like pure sabotage, it was misleading information to steer us away from the real part of the code that holds the solution…. That is of course, if we are kind enough to assume that he knew what/where it was in the first place.  In the end, he wasn’t smart enough to only lie about things that we could not verify.. now we know (we always knew, but now we have proof to back it) that he’s a liar, and I do not think that anyone will believe his lies anymore.

 

Enough talking about liars and drama queens, back to the 4.0 HEN solution… so what next? well, we now know that we can’t sign the file, so we can’t run our apps on 3.60+ (it can work on 3.56 though). What we will do is look for a different way, a completely new exploit that would allow the files we install to actual run on the PS3. We will also be looking for possible “signature collisions” and for that we will need the help of the community, hopefully there is a collision (same random number used twice) which will allow us to calculate the private key, and if that happens, then we can move forward with a release.

When will the “jailbreak” be released? If I knew, I’d tell you, but I don’t know.. I would have said in last november, then december, then before christmas, then before new year, etc… but as you can see, it’s impossible to predict what we will find.. we might get lucky and have it ready in a couple of days, or we may not and it will not be ready for another couple of months.. so all you need to do is : BE PATIENT (and please stop asking me about an estimated release date)!

I would like to thank the team who helped on this task for all this time and who never got discouraged, and I’d like to thank an anonymous contributor who recently joined us and who was instrumental in figuring it all out. We all believe that freedom starts with knowledge, and that knowledge should be open and available to all, that is why we are sharing this information with the world. We got the confirmation (by finding the public key used and verifying the signatures) yesterday and since sharing this information will not help Sony in any way to block our efforts in a future release, we have decided to share it with you.  We believe in transparency, we believe in openness, we believe in a free world, and we want you to be part of it.

If you want to know more about this ECDSA signature algorithm, I tried to explain it in a blog post here, also, you can read this interesting paper that explains it in detail, and you can also watch Team Fail0verflow’s CCC presentation that first explained Sony’s mistake in their implementation, which made custom firmwares possible.

 

Thanks for reading,

KaKaRoTo

 

Tagged with: