Posts tagged ‘Technology’
Gartner, Hype Cycles, and Big Data
Greetings all! While we’ve been super busy at PatternBuilders working on a destination application that we are all very excited about, doing some development work, talking with potential partners and prospects, AND not to mention the fact (but I will) that Terence and I are getting close to the finish line on our book, I came across this interesting article on Gartner’s hype cycle and the recent addition of big data to it.
Now, I am pretty sure that you all are familiar with Gartner’s magic quadrant methodology that essentially evaluates all the particular technology players in a specific industry across four quadrants: challengers, leaders, niche players, and visionaries. For those companies looking at vendors for a specific solution, the magic quadrant can help them understand how they stack up against each other. For the vendors, it’s an opportunity to take an objective look at the industry as a whole and understand what they do well and where they could be better. (more…)
It’s About Time: Series Data, Streaming, & Architecture
In previous posts, we have talked a lot about the PatternBuilders Analytics platform and streaming analytics. This platform is able to scale for huge amounts of data and stream results to the user as they are processed in real time. As mentioned before, we can do this because we have focused on time series analytics, making optimizations to our architecture that beat generalized MapReduce types of solutions by orders of magnitude. I’d like to discuss this focus and how it came about.
Why time series data?
Time series data is ubiquitous. It’s actually more difficult to think of an analytics question a user would be interested in that doesn’t involve time in some capacity. Even a non-numeric query like “Order the list of products by units sold” is almost useless without specifying a time period for which to sort. (more…)
MongoSF: Our Streaming Analytics Video is Now Available
As you all know, Tim and I spoke at MongoSF recently. Our session was focused on how to build a streaming analytics system with Mongo. For those of you who might have missed this post thread, here are the highlights (with the appropriate links):
- We wanted to make our beta version of PatternBuilders Social Media Analytics demo publicly available on the web.
- We looked at cloud-based deployments as a way to make this economically viable.
- As part of our move to the cloud, we made significant changes to PatternBuilders Platform architecture—which included MongoDB (a choice that the PatternBuilders development team is very happy with).
Our session was videotaped and I am happy to announce that it is now available on the 10gen site. You’ll notice that we got a lot of great questions. If, after viewing the video, you have some thoughts or questions please send them my way through comments or email—it may take me some time (we are, as Mary said in her last post, crazy busy right now), but I will follow up!
We love .NET
PatternBuilders sells a hosted cross platform streaming analytics platform that large companies use to do complex calculations and business process automation over very large data sets. So it was fascinating to read a recent post/troll from the CEO of a company that is writing yet another web based expense tracking system about how bad our technology and hiring choices were. Since we never like to pass up a good scrap – it seemed like a good time for a guest post from our lead server engineer Tim.
By Tim L.
I don’t really understand why David Barrett wasted time writing his rant on .NET programmers. Doing a minimal amount of research on what .NET is, what you can do with it, and how people use it would have completely invalidated his original premises. He makes a lot of statements regarding how “different” .NET is from everything else, how restrictive it is, and how no programmer with “attitude” would ever use it.
Well, judging by his criteria, I think I am a programmer with attitude. I have been programming since I was 9, starting out with Basic and then moving on to C++ for about 7 years. I don’t know about knife fighting, but I do play guitar in a death metal band on the side. Hopefully those are enough “attitude” credentials for David Barrett. I have tried a whole lot of different tools, and guess what? .NET & C# are my favorite tools for almost any problem. Ironically, the only things I would write using other toolsets would be either very simple/small pieces of code, or big software for companies that force me to use something else (usually Java).
(more…)
Data Privacy in the Age of Big Data: Ebook Coming Soon!
How the players, regulators, and stakeholders are changing the privacy landscape.
As Mary mentioned in the previous post, we have a big announcement to make: we are co-authoring a series of Ebooks for O’Reilly. The first in the series will focus on the data privacy landscape as it is today, including:
- The technological innovations driving big data and analytics, and how those innovations are forcing us to reexamine and often re-invent our ideas around data ownership, privacy and security.
- The rise of data collectors, aggregators, and marketplaces, their methods of capturing and monetizing data about you, and how that impacts your privacy.
- How the worldwide regulatory and legal landscape is evolving as governments try to balance the security and convenience benefits of these technologies against their potential for abuse.
- Why the music industry and other large copyright holders have become leading figures in the battle against consumer privacy.
- How governments use and abuse their citizens’ digital data.
People often ask why we spend so much time writing about digital privacy issues. First, as a company that is on the forefront of creating sophisticated tools to analyze digital data, we are acutely aware of the powerful technologies and techniques we—and companies in our industry—are developing. And as I wrote here, we think a lot about what we can do to ensure that our tools and expertise are used in ways that are ethical and positive. Helping our customers and the public be proactive about privacy issues is one of the best ways to do that. (more…)
Strata Conference Wrap-Up
There’s nothing like a great conference to get ourselves and the rest of the world galvanized for action and ready to take on the big data and analytics challenges that are coming our way. We will certainly be talking about it over the next couple of weeks and sharing with you some of our insights on the conference and the trends that we not only talked about ourselves, but heard from other Strata participants.
As you may already know, the conference was sold out. However, Strata is featuring videos of all the keynote sessions. When you have the time you may want to listen to the keynotes—I did and they were interesting and quite often, inspiring.
Terence, as you know, was a presenter on Tuesday, talking about “Retail: Lessons Learned from the Original Data-Driven Business and Future Directions.” Along with that, he was interviewed by Mac Slocum, O’Reilly’s Online Managing Editor about future big data technology trends, what the large retailers and online big data companies, such as Google and Amazon, can learn from each other, how what they do will trickle down to empower smaller businesses, and the enormous impact mobile will have on what we like to call the concept of the virtual person.
Stay tuned and get ready—we are going to spend quite a bit of time talking about analytics from a business, technology, and even “for the greater good” perspective!
A Deep Dive Into PatternBuilders Analytics Framework (PAF)
Today one of our Server Engineers is going to give you a deep dive on our architecture. As always on our blog, all of the data is simulated and all trademarks are the property of their respective owners.
Hello everyone! I am going to get fairly technical in this post and go over how PatternBuilders Analytics Framework (PAF) does what it does so well. As Terence has said in the past couple of posts, we have a new architecture that’s based around scalability, streaming, and ease of use. That’s not quite the whole story though; the development of this architecture was in fact driven primarily by performance. (more…)
Riding the Data Waterfall
Our new streaming analytics engine.
As promised, I am going to spend the next few posts discussing some of the new features in our analytics framework, otherwise known as PAF. This is our largest and most complex release so far. We are very proud of it—both in how far the framework has come and how closely it matches our vision of what a world class analytics system would look like when we started the company a few years ago.
One of my favorite features, and certainly the biggest change in this release, is that our analytics engine is now completely streaming based. I think that this, along with our improved ad-hoc analysis support, is going to improve our customers’ day-to-day to experience with both calculating and using analytics in their businesses. (more…)
Everything Old Is New Again
I had a discussion recently with a very talented but young product manager on the merits of SaaS (Software as a Service) and multi-tenancy versus traditional Enterprise Software. The discussion got a little easier when I gave him some history about how we got here. Yeah, I realize that knowing this history makes me ancient in programmer years.
SaaS could be defined as the combination of a new architectural model known as multi-tenancy with a payment model which has been around since the mainframe era — timesharing. In this post I will focus on the history of timesharing. In the next post I will discuss multi-tenancy.
SaaS re-introduced the hardware/software rental model known as timesharing to the world. For those of you who are young enough to think that Paul McCartney was always a solo act it may seem shocking, but yes people where renting software and data centers decades before Salesforce.com was founded.
(more…)
Big Data Big Analytics 