Trend #1: Digital Advertising Comes of Age and Continues to Propel the Growth of Data Brokers and Markets

By Mary Ludloff & Terence Craig

Authors’ Note: As promised, this is trend number one in our take on the top ten privacy trends or the more things change, the more they stay the same. Although similar to David Lettermen’s Top Ten lists, our list is not as funny (unless we get points for sarcasm). We would like to remind you that sifting through the media coverage, books, blog posts, and research studies has been no easy task and trying to understand how all of it fits into the larger privacy landscape has been even harder. Added to that, our list is sure to elicit a litany of trends that we missed. We certainly hope that it does and welcome your input in our comments section! Now, on with the trend number one!

It is safe to say that continued media attention (mostly negative) has not had an impact on the rising use of third party tracking mechanisms to collect personal information. According to the Web Privacy Census, an undertaking sponsored by the Berkeley Center for Law and Technology, which measures and benchmarks Internet tracking over time:

• The number of cookies discovered on the top 100, 1,000, and 25,000 websites are all significantly increasing. For example, in May of 2012 5,795 cookies were found on the top 100 sites and in October of 2012 the number reached 6,485.
• The percentage of cookies set by a third party host was 84.7%. In other words, most of us are being tracked by a host of parties that we have no data collection agreement with.
• The top trackers were BlueKai (the largest online auction marketplace), Rubicon Project (one of the larger real-time bidding systems that sells ad space on web pages), and Adnxs (the advertising exchange for advertising exchanges).

Privacy and Big Data—Update on the Update

By: Mary Ludloff and Terence Craig

We’ve been promising an update to our book, Privacy and Big Data, since the just pre- and mostly post-Snowden era. When we proposed and wrote the book it was to fill a void. At the time, there was a lack of mainstream attention to the issues of privacy by the media and a lack of understanding of the issues and implications we all face in the digital world. As tech veterans of long standing, we have seen our world transformed for better and for worse by our industry. Much of the “worst” we chronicled in our book and at the time its release, many relegated our book and ourselves to the “foil hat” and “black helicopter” brigade.  Yes, that was a “we told you so” but we promise it’s the last one.

Then came the Snowden revelations which raised its own hailstorm of media attention, information, misinformation, and disinformation (primarily by our government officials, legislative leaders, and the President as well as the Prime Minister of the UK) on exactly how our data was being collected and what it was being used for. Cynics though we are, we wondered if digital privacy issues had finally reached a tipping point, that we would have a national conversation about civil liberties, how to fix FISA, what is the acceptable collection and use of our data by commercial and government entities, and moving forward, how our liberties and data could be protected from corporate and government spying.

Privacy v Security, Transparency v Secrecy: The NSA, PRISM, and the Release of Classified Documents

By Mary Ludloff

Privacy, Anonymity, and Judicial Oversight are on the Endangered List

An age old debate has once again reared its very ugly head due to whistleblower Edward Snowden’s revelations about NSA surveillance, PRISM, and the astounding lack of any rigorous oversight on the NSA’s vast data collection apparatus.  While PatternBuilders has been incredibly busy, in our non-copious amounts of spare time Terence and I have also been working on our update to Privacy and Big Data (which is undergoing another rewrite due to new government surveillance revelations that for a while happened hourly, then daily, then weekly but certainly are far from over). It’s important to note that pre-revelations  our  task was already herculean due to mainstream media’s pick up on “all stories related to privacy” (a good thing) that often missed the mark on the technical side of the house (we often find ourselves explaining to non-techies just what meta data is which usually happens after someone on CNN, Fox, NBC, ABC, etc., butchers the definition) or got tripped up by the various Acts, Amendments, state laws, EU Directives, etc., that apply to aspects of privacy.

Over the last few weeks as details about PRISM emerged, it’s become clear to me that main street America may still not understand the seismic shift that big data and analytics brings to the privacy debate. Certainly the power of big data and analytics has been lauded or vilified in the press—followers of our twitter feed are used to seeing the pros and cons of big data projects debated pretty much every day. We’ve (Terence and I) talked and tweeted about privacy issues as it applies to individuals, companies, and governments. Heck, we even wrote a book about privacy and big data.

Big Data and Science: Focus on the Business and Team, Not the Data (Part 3 of 3)

By Mary Ludloff

Let me tell you a little secret: I always know when I am talking (and working) with a company that has successfully launched big data initiatives. There are three characteristics that these companies share:

1. A C-level executive runs the “[big] data operations.”
2. The Chief Data Officer (even if they are the CIO) has a heavy business/operations background.
3. The data team is focused on the “business,” not the data.

Did you notice that technology and data science are not reflected in any of the characteristics? Some of you may consider this sacrilege—after all, we are operating in a world where technology (and I happily work for one of those companies) has changed the data collection, usage, and analysis game. Colleges and universities are now offering master degrees in analytics. The role of the data scientist has been pretty much deified (I refer you to Part 1 of this series). And we all need to be very worried about the “talent shortage” and our ability to recruit the “right analytical team” (I refer you to Part 2 of this series).

Yes—technology has had a tremendous impact on how much data we can collect and the ways in which we can analyze it but not everyone needs to be a senior computer programmer. Yes—we all should strive to be more mathematically inclined but not all of us need Master’s or PhD’s in statistics or analytics. Yes—some companies, based on their business models, may have a staff of data scientists but others may get along just fine without one (with the occasional analytics consultant lending a hand).

Privacy and Big Data: Speaking at Strata East (NYC), Book Update, and Upcoming O’Reilly Webcast

By Mary Ludloff

There are times when Terence and I look at each other and say, “What on earth were we thinking?” And this is one of those times! PatternBuilders is crazy busy right now putting out release 3.0 of our Analytics Platform (the secret sauce for our analytics applications that we like to call data-science-in-a-box), ramping up on a funding round, working with partners on a University of Sydney research project on the impact of social media on a company’s stock price (a really fun project and a post about it is in the works), and, of course, supporting customers and prospects on their big data initiatives. So… since we did not have enough to do (sarcasm on), we decided it was time to update our book, participate in a pre-Strata East webcast, speak at the Strata Conference and the MongoDB User Group (that is collocated with Strata) in New York City! In the words of the immortal Bette Davis in All About Eve (and ever so slightly revised):

“Fasten your seat belts, it’s going to be a bumpy night ride!”

Really, what were we thinking?????

Privacy, Big Data, Civil Rights, and Personalization Versus Discrimination: When does someone else’s problem become ours?

By Mary Ludloff

There has been a great deal of media attention on the benefits of big data (just look at our @bigdatapbi twitter stream) lately. Certainly, PatternBuilders has been busy helping financial markets become more efficient, working with data scientists on various research projects, as well as helping other businesses with their big data initiatives. In fact, there are a number of companies (like ours) that are making significant strides in reducing the costs associated with legacy big data systems, helping to move big data out of the early adopter phase and into the mainstream. But as technology innovates, there is usually some “bad” thrown in with all that good. Such is the case with big data and privacy.

Two thought provoking articles on privacy were published this month—both considering privacy through a civil rights prism. In “Big data is our generation’s civil rights issue, and we don’t know it,” Alistair Croll states that:

“Personalization” is another word for discrimination. We’re not discriminating if we tailor things to you based on what we know about you — right? That’s just better service.”

Privacy and Big Data: An Update is in the Works and We Want Your Input!

By Mary Ludloff

In July of 2011, Terence and I were doing our day jobs (which in startup-land is way more than 8 hours) and working on “Privacy and Big Data” during the evenings and weekends. It was, by all accounts (at least according to our friends and family), a lovely summer and we missed most of it! We spent “our” July, August, and September combing through research, studies, media reports, blogs (we stopped counting the number of pages when we got to 4,000) to put together a book that was our humble attempt to cover the full spectrum of the privacy landscape:

• Our rights and expectations of privacy historically and in the digital age.
• The current “state” of privacy regulations here (U.S.) and abroad.
• The players (governments, industries, movements, and companies) that have a stake in the privacy debate and their often contentious and competing agendas.

Getting Into a Privacy Identity Innovation (pii2012) Frame of Mind: Will We See You There?

By Mary Ludloff

As you all know, privacy is one of my favorite topics. And when you’re talking or blogging about privacy, it almost always comes back to personally identifiable information (pii) which just happens to be the focus of the Privacy Identify Innovation Conference that will be held in downtown Seattle, May 14-16 (otherwise known as pii2012). Natalie Fonseca (@TechPolicy) is the co-founder and executive producer of the conference (now on its “third edition”), and she has assembled a group of speakers and sessions that make privacy-geeks (like myself) giddy with anticipation:

“pii2012 Seattle will explore how to protect sensitive information while enabling new technologies and business models. The focus isn’t just on ensuring regulatory compliance. It’s about developing a forward-looking, strategic approach to avoiding risk while advancing innovation.”

Confessions of a Privacy Junkie (and a list of my favorite privacy resources!)

By Mary Ludloff

The holiday season is upon us and we are being inundated (happily) with best and worst lists as well as predictions for 2012. Terence and I will reveal our own lists soon but I thought that I might take this post to chat about our book (Privacy and Big Data) and the evolution of a privacy junkie (me). But before I begin, a note to our regular readers: I know that I promised a part 2 in my series on McKinsey and location tracking and it’s coming… in early 2012 (yes, I have taken poetic license with scheduled dates!).

As you all are far more aware than you might want to be due to our incessant plugging, we (Terence and Mary) wrote a book on Privacy and Big Data this year. I have always been concerned about privacy issues and thought that I was well informed on the subject so why not co-author the book? How much work would it really entail? After all, this is a topic that Terence and I talk (and argue) about pretty much every day so all we needed to do was collate our combined knowledge and voilà, a book is born! Six months later, after reading through thousands and thousands of pages of research, books, articles, and posts on the topic of privacy, I can safely say that I am now a privacy junkie of the highest order (I suspect Terence is as well but I digress).

Data and Technology Have No Moral Compass: But that does not mean that we get to abdicate all responsibility.

By Mary Ludloff

I do not consider myself an idealist and I would not call myself naive. That being said, as Terence and I engaged in research for our book, Privacy and Big Data, there were moments when I threw up my hands and said, “Really?” Certainly, the recent spate of articles on surveillance technologies and how governments around the world are buying and using those technologies to, for want of a better term, spy on its citizens gave me pause.

Don’t get me wrong—I know these technologies exist. I am also very aware that the regulatory environment does not really address what devices or applications built on top of these technologies can do. The reality is that companies like Datong sell “intelligence solutions” to the military, law enforcement, and intelligence agencies around the world. Recently, an article in the Guardian revealed that: