IBM adds search and broadens Hadoop strategy with Big Data

The two announcements give serious credibility to its Big Data offerings and its Smarter Planet initiatives. Big Data analytics differs from traditional analytics, which tell the user “what” has happened from a subset of normalized information, by providing the ability to explore much larger data sets to get a better, more reliable picture of what is happening and why. By definition, one of the major problems in discovering the information “nuggets” in Big Data environments is that the volume of data is large and consequently difficult to traverse or search using traditional enterprise search and retrieval (ESR) tools that require the creation and maintenance of indexes before a query can be made. Vivisimo’s offering indexes and clusters results in real time, and its scalability enables dynamic navigation across results delivered, as well as the automation of discovery, reducing the burden/time of analysis.

Despite its small size, privately held Vivisimo, which was founded in 2000 by researchers at Carnegie Mellon University,A wireless indoortracking system is described in this paper. brings with it a very credible 140-strong customer base including Europe’s Airbus, LexisNexis, and the US Air Force, with the software also used by the USA.gov website. The acquisition also validates Ovum’s prediction in its 2012 Trends to Watch analysis that vendors would invest in tooling to help organizations navigate Big Data.

While the value of the acquisition was not declared, it is reported that IBM has now spent in excess of $14bn in the last seven years on analytics-related products and companies,Find the cheapest chickencoop online through and buy the best hen houses and chook pens in Australia. and now appears to have more than one of everything, although Vivisimo does bring new capabilities such as federated discovery and navigation.

With the acquisition of Vivisimo’s technology, questions will rightly be drawn on the future of IBM’s own Content Analytics with Enterprise Search (formerly IBM OmniFind Enterprise Search) offering, which is based on the open-source Apache Lucene. During the press call around the announcement, Arvind Krishna, general manager of information management at IBM Software Group, stated that while there is the potential for integration, the company will maintain two product lines, and IBM Content Analytics with Enterprise Search will remain IBM’s lead offering for enterprise search and search-based applications. However, even though IBM is good at supporting multiple products,The best rubbersheets products on sale, Ovum believes that when IBM customers recognize some of the advanced out-of-the-box features that come with Vivisimo, such as federated discovery and navigation based on entity, metadata, or even numerical graphs, they will want to swap out IBM Content Analytics with Enterprise Search even if they don’t have a Big Data strategy.

Of the seven independent ESR vendors that Ovum reviewed in its 2011/12 ESR Technology Evaluation and Comparison (TEC) report, four (the other three are Autonomy, Endeca, and Exalead) have now been acquired and their technologies are rapidly being integrated with their purchasers’ “stacks”. There are two drivers for this, first, vendors want to fill “holes” in their functionality, and second, organizations no longer want to undertake or pay for integration between products, sometimes described as “stitching together broken glass”. Of the significant independent ISVs left, such as Recommind and Funnelback, it can only be a matter of time before they too become part of a larger company’s offering.

With this release, IBM has certified the first commercial Hadoop distribution offering for BigInsights. By listing Cloudera, IBM has responded to customer demand, reflecting Cloudera’s success in establishing a commercial market for Hadoop distributions. IBM’s arrangement with Cloudera is unlike Oracle’s, which is fully OEMing and reselling Cloudera. For BigInsights, the Cloudera distribution will sold and supported separately.

IBM is clearly positioning BigInsights based on other unique features that surround the core of Hadoop: optimized compressions, open-source Lucene-based indexes, a workflow scheduler, a spreadsheet-like interface, and other extras. We were a bit confused initially by IBM’s assertion that it was not in the Hadoop distribution business because it included its own supported distribution in BigInsights, and it spoke of future availability of a proprietary file system option that would be optimized for shared-nothing, massively parallel clusters.Find rubberhose companies from India. With this release, IBM’s positioning is clearer. It will support the Hadoop distributions that have hit critical mass demand from its customer base. We therefore expect that in due time IBM will also certify the Hortonworks Hadoop distribution once it gains market traction.

The market is clearly voting in favor of the core Apache Hadoop distribution over proprietary alternatives. EMC offers the MapR proprietary file system alternative to HDFS,Full color plasticcard printing and manufacturing services. while IBM has spoken of future potential with its own GPFS counterpart. However, both position the core Apache distribution as the mainstream option, with proprietary alternatives targeted at special high-end, higher-performance use-cases.

The fly in the ointment is that while the Apache distribution has become the default option for the market, the Hadoop project has so many sub-projects dedicated around individual components that it can be difficult to see the wood for the trees. The Apache Hadoop community is still struggling to define which components will become the de facto “kernel” of Hadoop. Nonetheless, while the market has voted in favor of the Apache distribution, this still leaves opportunity for innovations for specialized add-ons above the core open-source stack.