Tech Threads

How Google news able to group similar news together

Google news uses clustering machine learning technique to group similar kind of news or articles together.  Interestingly, they don’t have thousand news editors on trunk instead use the clustering technique to forms groups of similar data based on the common characteristics. Mahout is a machine learning software from Apache community that applications leverage to analyse large sets of data.  Before invention of Mahout, it was too complex to a analyse large sets of data. Mahout extensively utilize Apache Hadoop to get the power parallel and distributed computing. Three machine learning techniques that offers by Mahout.

  • Recommendation
  • Classification
  • Clustering

Clustering is not group data into an existing set of known categories.  This is particularly useful when we are not sure how to organize the data in the first place.  Google news uses this powerful technique to make change the ever-changing stream of news and articles from around the world to enabling us keep update with latest events around the globe.

Recommendation technique uses user information + community information to deliver what type of product/service etc we would like or prefer mainly when we are browsing e-commerce sites.

Page: 1 2

Recent Posts

Transferring real-time data processed within Apache Flink to Kafka

Transferring real-time data processed within Apache Flink to Kafka and ultimately to Druid for analysis/decision-making.… Read More

3 weeks ago

Streaming real-time data from Kafka 3.7.0 to Flink 1.18.1 for processing

Over the past few years, Apache Kafka has emerged as the leading standard for streaming… Read More

2 months ago

Why Apache Kafka and Apache Flink work incredibly well together to boost real-time data analytics

When data is analyzed and processed in real-time, it can yield insights and actionable information… Read More

3 months ago

Integrating rate-limiting and backpressure strategies synergistically to handle and alleviate consumer lag in Apache Kafka

Apache Kafka stands as a robust distributed streaming platform. However, like any system, it is… Read More

3 months ago

Leveraging Apache Kafka for the Distribution of Large Messages (in gigabyte size range)

In today's data-driven world, the capability to transport and circulate large amounts of data, especially… Read More

5 months ago

The Zero Copy principle subtly encourages Apache Kafka to be more efficient.

The Apache Kafka, a distributed event streaming technology, can process trillions of events each day… Read More

6 months ago