June 27, 2017

Archives for October 2016

The AT&T Deal Is About the Data

Most of the mainstream media coverage of the proposed AT&T acquisition of Time Warner has missed an important risk. Much of the discussion has focused on the potential market power the combined entity would have to raise prices, limit choice or otherwise disadvantage consumers.

A primary motivation for the deal, however, as readers of Freedom to Tinker well understand, is the desire to access more and deeper data about consumer behavior. The motivation to combine companies is not monopolistic control, but rather a timely effort to become a player in the lucrative, $77 billion world of targeted digital advertising, now controlled by Google and Facebook.

Some media, especially those covering the FCC and FTC, have begun detailing the data privacy issues raised by the deal. Hopefully, mainstream media will soon follow suit.

Here are some links:

BNA: FCC Privacy Rules Could Hamper AT&T-Time Warner Data Mining

Inside Sources: Mega Mergers Like AT&T-Time Warner are Becoming a Problem for Privacy Regulation

Bloomberg: Privacy Rule Imperils Data Riches as AT&T Pursues Time Warner

Fortune: Media Companies Want U.S. to Force AT&T-Time Warner to Share Customer Data

Neophilia and Human Nature

In the spring of 2012, I attended the memorial service for John McCarthy, a computer science founding father, at an auditorium on the Stanford campus. Among the great and good anecdotes told about this great and good guy was the mention of how McCarthy, more or less in around 1961, invented time-sharing—which, as was pointed out, is what is now called cloud computing. The attendees at the memorial service gave small rueful laughs of recognition; other incarnations of the same idea have long cropped up from the 1960s onward, among them client/server architectures of the 1990s as well as The Long Now Foundation’s Danny Hillis’s notion of computing as a utility you pull in from the wall.

In 1987, when I wanted my then-boss to pay for a kind of gray-market Internet access (I had hunted down a Net route-around, as in those explicitly non-commercial days only academics and government were supposed to have Net access) he harrumphed and said, “I don’t want to pay for your hanging out online and flirting.” Even then, wasting time on screens was a known Thing.

Before there were Arab Spring and Twitter, there were Fidonet and Serbia. Before there was Facebook, there were Usenet, Compuserve, Plato, DECnet, Minitel. 1960s-era FCC Commissioner Nicholas Johnson often warned about the losses to privacy when big databases would be tracking and storing information about everyone, and sharing that information with each other. He was also concerned about the losses of privacy when video would make it possible to monitor people in public places.

As long as there have been electronic computation and communications, humans do what they have always done, their desires and drives are eternal. People want to communicate and flirt; businesses want to find competitive advantages; governments want to keep track of internal and external threats. Technology and its implementations may change but people do not. [Read more…]

Bitcoin is unstable without the block reward

With Miles Carlsten, Harry Kalodner, and Matt Weinberg, I have a new paper titled On the instability of Bitcoin without the block reward, which Harry will present at ACM CCS next week. The paper predicts that miner incentives will start to go haywire as Bitcoin rewards shift from block rewards to transaction fees, based on theoretical results that closely match up with findings from our new Bitcoin mining simulator.

Bitcoin provides two incentives for miners: block rewards and transaction fees. Currently the vast majority of miner revenues come from block rewards, but in the long run they will come primarily from transaction fees as block rewards dwindle. This design decision has been discussed a lot, but in terms of monetary policy and hardly ever in terms of security. There has been an implicit belief that the transition to transaction fees will not affect the security and stability of the block chain, and in particular that it is immaterial whether miners receive (say) 25 bitcoins as a fixed reward or 25 bitcoins in expectation via transaction fees.

We reexamine this assumption in our paper, and our findings make disturbing news for the future security of Bitcoin and many other cryptocurrencies. Our key insight is that with only transaction fees, the variance of the miner reward is very high due to the randomness of the block arrival time, and it becomes attractive to fork a “wealthy” block to “steal” the rewards therein. [1]


The figure shows a scenario where forking might be more profitable than extending the longest chain. See the paper for a full explanation.

Here’s how things could go wrong. Due to the possibility of profitable forking, the default strategy is no longer best; we lay out a menagerie of interesting and bizarre strategies in the paper. The most worrisome is “undercutting,” where miners capture as little of the available transaction fees as they can get away with, leaving the rest in the pool as an incentive for the next miner to extend their block rather than a competing block.

We also show rigorously that selfish mining gets worse when block rewards are replaced by transaction fees, motivated by the following intuition: if you happen to mine a new block just seconds after the last one was found, you gain nothing by publishing, so you might as well keep it for selfish mining in case you get lucky. The variance in transaction fees enables strategies like this that simply don’t make sense when the block reward is fixed.

If miners switch to these deviant strategies, the blockchain will be much less secure because of the mining power wasted due to constant forking, undercutting, and withholding of found blocks.

We derive most of our results in two separate ways: analytically, i.e., using game theory, and with a new mining simulator that we created. This gives us added confidence in our findings. For example, in one setting, the theory predicts a rather grotesque equilibrium involving on the Lambert W function, with the proof running to several pages. Sure enough, in our simulations of the same setting, the Lambert miner does best. We hope that our analytical techniques as well as our simulator will be useful to other researchers. We have made the simulator code open-source.

What is the impact of our findings? The Bitcoin community will probably need to respond to this problem in the long run, potentially via a fork, to discourage deviant strategies. We aren’t predicting that deviant strategies will arise in the short term, and there is a long runway for mitigation steps to be rolled out. The fact that blocks have filled up due to their 1MB limit decreases the variance of transaction fees between different blocks, and this mitigates the problem somewhat, although it is far from a complete and satisfactory solution. For example, at the time of writing our paper, the previous 1000 blocks included per-block transaction fees ranging from 0.03 BTC to 4.51, with a mean of 0.49 and standard deviation of 0.25 (over half the mean!). So simply maintaining the block-size limit probably won’t resolve the underlying issues.

At a deeper level, our results suggest a fundamental rethinking of the role of block rewards in cryptocurrency design. The prevailing view is that the block reward is a necessary but temporary evil to achieve an initial allocation of coins in the absence of a central authority. The transaction-fee regime is seen as the ideal steady state of the system. But our work shows that incentivizing compliant miner behavior in the transaction fee regime is a significantly more daunting task than in the block reward regime. So perhaps designers of new cryptocurrencies should make the block reward permanent and accept monetary inflation as inevitable. Transaction fees would still exist, but merely as an incentive for miners to include transactions in their blocks.

One final point: there is a science of designing economic incentives so that rational players will behave in a desired way, and it’s called mechanism design. Creators of cryptocurrencies (as well as creators of applications such as the DAO) are essentially doing mechanism design. But mechanism design is hard, and our paper is the latest among many to point out that the mechanisms embedded in cryptocurrencies have flaws. Yet, sadly, the cryptocurrency community is currently disjoint from the mechanism design community. That is why I’m thrilled that mechanism design expert Matt Weinberg, who’s behind all the sophisticated theory in our paper, is joining Princeton’s faculty next semester. Expect more research from us on the mechanism design of cryptocurrencies!

[1] The problems uncover arise not because transaction fees may arrive erratically, but because blocks inevitably arrive unpredictably.  We model transaction fees as arriving at a uniform rate. The rate is non-uniform in practice, which is an additional complication. This is a theme throughout our paper: we show that undesirable behaviors will arise even in simplified, “clean” models. This is bad news both because we think things will probably be worse in practice and because we want cryptocurrency mining games to be analytically tractable. Our work shows that in a transaction-fee regime, predicting behavior will be fiendishly complex.

Update: see also Bryan Ford’s response to this post (and paper).