Slush’s bitcoin mining pool hacked

SlushLogoIt seems that hackers have decided to start to target bitcoin related services in earnest as the latest organisation to fall prey is the very popular Slush’s bitcoin mining pool that usually resides at https://mining.bitcoin.cz/ .

Slush’s pool is the oldest mining pool and it started with a forum post on November 28, 2010 when the pool operator (Slush) literally invented pool mining when he suggested that “Join poor CPU miners to one cluster and increase their chance to find a block!”.    While the suggestion was quite controversial at the time Slush wrote the first mining pool software (Called a “Cooperative miner” at the time) and the rest is history.

According to Slush he first noticed something was up when someone reset the password to his OVH Manager account at his hosting provider OVH.CO.UK Web Hosting Solutions.

What ensued was a short battle between the perpetrator and Slush resetting passwords in attempts to gain control of the pool.  While Slush does not have any evidence yet he says “So far it looks like yet another inside job, like Linode two years ago. Or attackers found some shortcut how to gain access to Manager without confirming the request from the email.”.  Slush later goes on to say “For now I fully blame OVH for this issue.”

Slush says he has now successfully managed to isolate and move the Stratum servers ( stratum.bitcoin.cz, stratum2.bitcoin.cz and stratum3.bitcoin.cz) to Amazon EC2 instances so as not to waste any hash power but as there is no safe database server the shares are not currently being recorded and Slush says “Because database isn’t running and shares are not stored, I’ll spread blocks mined during database outage to miners who’ll continue mining on the pool since the database will be up again.”

Slush also says that the mining pool will be back to normal operation soon after he fully migrates from OVH to Amazon EC2.

The first post on BitcoinTalk is available here or quoted below.

The pool has been hacked. Fortunately I noticed it fast enough, so I made database snapshot seconds before attackers overtake the database machine. I lost some amount of bitcoins, but I’ll be able to recover it from my pocket. For now I’m evaluating what’s next to do, because all machines in OVH has been compromised and they cannot be trusted anymore.

Full story:
Today at 3pm UTC I noticed that somebody succesfully resetted the password to OVH manager, the place where servers can be managed, restarted to rescue mode etc. I promptly resetted the password at OVH to something different and I also changed password on my email account and checked that there’re no other active connections to my mailbox. I have to say that my mailbox is secured by OTP passwords and I take physical security very seriously, so nobody other had an access to my mailbox. I known that password-reset feature is quite popular attack vector, so I made everything possible to prevent it to happen.

By changing the password at OVH, all other sessions using the old credentials are automatically kicked from the Manager. I also cross-checked that nothing wrong happen to the servers at this time. Unfortunately I didn’t find a way how the attackers got access to Manager, so I asked OVH support to provide some additional information and restrict Manager access to my IP range.

That’s no surprise that OVH didn’t respond to this ticket for hours, but at 11pm UTC I realized that there’s another succesful password reset at OVH. This is complete mystery to me, because I’m aboslutely sure that nobody else had access to my mailbox and the email with reset link has been untouched (unread, not deleted). I’d say that attacker won’t bother by changing status of the email to “unread”, but he’d delete the email instead.

This time I realized that the attacker resetted the machine with the wallet to rescue mode, which means that I lost the control to this machine. I was still succesful by logging into the database and I took the snapshot of database and transferred it to safe location. Few seconds since the migration finished, attackers restarted all remaining machines to rescue mode.

So far it looks like yet another inside job, like Linode two years ago. Or attackers found some shortcut how to gain access to Manager without confirming the request from the email. I don’t know what’s worse option. I’ll investigate this issue in detail later and I hope OVH won’t close eyes to this.

I can recover the pool to the normal operation tomorrow.

Edit 01:38 UTC: Stratum servers are running on safe servers at Amazon. Mining works for now. I’ll setup new database and webserver on trusted machines in few hours, so the pool will be back in full operation.Slush – BitcoinTalk

Neil Fincham, the MineForeman has over 20 years experience in the computer industry and runs the MineForeman mining operation for the co-op members.

Tagged with: , , ,
Posted in Bitcoin, Mining, News, Pools
One comment on “Slush’s bitcoin mining pool hacked
  1. DBG says:

    Every person who has dealt with servers likely knows the old saying, “cheap, fast, secure… pick two”. While most providers would like to think (or at least claim) that they can offer all three across the board, it’s simply not the case for the vast majority. Of course OVH UK/France has a long history of providing the first two and even as a one time business customer of theirs, the support is horrid. I personally don’t believe it’s an inside job (assuming this is referring to an OVH staff member), however when I used them not that long ago, I found their security to be a joke and actually had to fight with them to try and disable their own control panel (which Slush talks about), as I simply did not trust it and wanted a completely hands-off, unmanaged dedicate server (well there were 3 at the time) in wish I could lock down as I saw fit. Long story short(er), they were not able or willing to do so and I gladly took the couple hundred dollar lost in advanced paid services to get out of there.

    Between this event and the DDoS attacks of last month that seem to hit a lot of pools and Bitcoin-related sites, I’m wondering if this is just the start. Unsurprisingly no person or group has claimed responsibility (I mean, what would they call themselves, “The Bitcoin Liberation Army”?) because I’m sure they are involved in some way with Bitcoin themselves (as in users who are aware of their value; I’m not suggesting that Avalon was responsible for the attack on BFL’s servers; which some people suggested, both jokingly and seriously). I’ve been lucky enough to pick Slush’s brain in private correspondences and although it doesn’t need to be said, he is a very smart guy, sadly he seems to be continuously burned by those who require the highest amount of trust. Events like this have really made me consider creating my own hosting company, however even though I have great connections worldwide, I really don’t think I could offer a cheap enough price, and there are already a lot of excellent “fast and secure” (remember, you only get to pick two =p) established companies. What I do now is help friends/projects/etc move to Amazon’s EC2 platform (which I wasn’t surprised to see Slush had done for the time being), as that’s where I have been for several years and have a self-coded suite of tools that allows me to keep costs low (Amazon’s built-in services are improving, but there are still some issues that could end up costing you a few dollars here and there in unneeded charges, which add up over time). I’ll end my post here as you have been generous enough to not ban me (well, to date =p) for my extensively long replies; also I’m suppose to be on bedrest since my return from the hospital and I don’t think being glued to my laptop is what they had in mind =).

    – Tony R./D(ee)BG

1 Pings/Trackbacks for "Slush’s bitcoin mining pool hacked"

%d bloggers like this: