mapping the US blogosphere

From MediaCloud :

One thing that we can use quantitative text analysis for is to get a sense for the overall landscape of a set of blogs. The following map of popular blogs in the U.S. (the top 1000 blogs according to Bloglines) gives a good sense of what topics people write about on popular blogs and how those topics relate to one another:

This map is generated by comparing the similarity of the top 100 words of each of the blogs in the popular blogs set during the first five months of 2011. Each dot represents a blog. We generate the cosine similarity of each blog to each other blog in the set and then designate the 3000 blog pairs with the highest similarity scores as being similar to one another. The map is then laid out using a force based mechanism, trying to push apart all of the nodes, with the 3000 similarity pairs acting as forces keeping those pairs together. So blogs that are similar to one another end up closer to one another in the map. The color of each node is the result of using the same cosine similarity scores to group the blogs into 25 separate clusters according to the similarity of the blogs’ content.
This map tells us that this U.S. popular blogs (to the degree they are represented by the particular Bloglines sample) can be grouped into three big meta clusters: Crafts, News, and Technology. Of these, the Craft meta-cluster is the biggest, and the biggest single cluster with 137 blogs is the big ‘love’ cluster (cluster labels are automatically generated as the most common word within the cluster’s member blogs). This is the most general cluster in the map, as indicated by the fact that it sits closest to the center of the map of any cluster. It is a mix of general crafting blogs and general personal life blogs. A few of the blogs include: [ Wee Wonderfuls ] [ Loobylu ] [ A Dress A Day ] [ Sea of Shoes ] [ PostSecret ] [ still me ] [ Loldogs, Dogs 'n' Puppy Dog Pictures - I Has A Hotdog! ] [ six and a half stitches ] [ CrazyAuntPurl ]. The word cloud for the ‘love’ cluster is strongly dominated by the title world, with mostly crafting words among the next most common:

The next largest cluster, with 44 blogs, within the crafting meta-cluster is the ‘knit’ cluster, which is exactly what it sounds like. According the clustering run, half as many blogs within the Bloglines top 1000 feeds focus on knitting as do on politics (82, which we’ll discuss below). A few of the blogs in this cluster are: [ Lolly Knitting Around ] [ SO MUCH YARN, SO LITTLE TIME! ] [ Grumperina goes to local yarn shops and Home Depot ] [ Twist Blog ] [ turkey feathers ] [ Hello Yarn ] [ Sheri at The Loopy Ewe ] [ Yarn-A-Go-Go ]. The word cloud for the cluster is exactly what one would expect a knitting word cloud to look like, with very strong focus on knitting and differentiation from other clusters:

Another prominent cluster within the crafting meta-cluster is the ‘recipe’ cluster, with 29 blogs. The blogs in this cluster include: [ (Inside A Black Apple) ] [ This American Life ] [ Baking Banter ] [ Cooking For Engineers ] [ Cheap Healthy Good - Frugal Recipes and Food Tips ] [ CRAFT ] [ smitten kitchen ] [ Accidental Hedonist ] [ not martha ]. The word cloud for this cluster consists almost entirely of directly food related words:

Almost all of the blogs and the top words in this cluster are on the topic of food. But the inclusion of This American Life, a radio show not focused at all on food, is a typical artifact of a content-based clustering process. In this particular case, This American Life posts a short summary of each show each week at its RSS feed. Two of the twelve weekly stories posted to its feeds for the time period of this clustering run are about the original recipe for coke and about the Onion, respectively. Because the rest of its stories are so diverse, those two stories, especially the recipe story, are enough to throw it into this recipe cluster.
The news meta-cluster is dominated by the ‘obama’ cluster of news sources. The ‘obama’ cluster has 82 feeds that are a a mix of general news, soft news, and political news in blogs and in mainstream blogs, including: [ Calculated Risk ] [ All Salon ] [ nybooks.com: Latest articles ] [ youtube :: most viewed videos - today ] [ Right Wing Nut House ] [ Overheard in New York ] [ NYT > Arts ] [ NYT > Opinion ] [ Yahoo! News: Top Stories ] [ Waiter Rant ]. This cluster has such a diversity of sorts of blogs because even this diversity of types of news gets swamped by the difference between any kind of news and the sorts of content in the crafting or technology meta-clusters. The word cloud for the ‘obama’ cluster shows that the content in the cluster is mostly dominated by politics, regardless of the diversity of the sources. In fact, the cloud acts as a pretty good description of political news coverage for the six months covered by the clustering run, with top words including not only the obvious ongoing ‘obama,’ ‘republican,’ and ‘democrat’ topics but also more specific topics dominating particular news cycles, like ‘egypt’, ‘libya,’ and ‘japan.’

The final meta-cluster is on technology, dominated by the ‘google,’ ‘app,’ and ‘iphone’ clusters. In this case, the k-means clustering that picks out cluster members (and assigns colors in the map) is able to subtly differentiate between these computer and Internet topics even though they are very closely related, even to close human inspection, but the mapping process (which assigns the position of the blogs on the map) is not able to differentiate among these blogs as well, leading to the dense clump of blogs on the right of the map.
The google cluster mostly consists of blogs that focus on topics directly related to google, including search engines (‘keyword’), search engine optimization (‘seo’), and online advertising (‘adsense’). The blogs in this cluster include: [ TEDTalks (video) ] [ Google Earth Blog ] [ Techdirt ] [ Coding Horror ] [ Google Sightseeing ] [ Daily Blog Tips ] [ Small Business Marketing Blog from Duct Tape Marketing ] [ Adverblog ] [ Gmail Blog ]. There is potential for this cluster to contain some artifact members, since ‘google’ is such a common word on the Internet and is often used for meta-comment (eg. ‘use google to search this site…’). But in fact, every member of this cluster is actually substantively focused on Google or on the issues closely related to Google described in the cluster word cloud:

Finally, the iphone cluster includes blogs that focus on personal technology in general and iphones and ipads in particular: [ Pogue's Posts ] [ Going My Way ] [ ThinkGeek :: What's New ] [ AnandTech Article Channel ] [ MobileCrunch ] [ TechCrunch Japan ] [ Web Designer Wall - Design Trends and Tutorials ] [ Apple Hot News ] [ Cool Hunting ]. Even though the app cluster above includes desktop computing platforms (‘microsoft’ and ‘mac’), blogs about those platforms are only strongly represented in that general cluster. Blogs in the iphone cluster, in contrast, focus strongly on iphones and gadgets, creating a strongly separated cluster:

Apple Ignores MAC Defender Malware

By Paul Mah

Apple (NASDAQ: APPL) is seeing a spike in malware infections from a Trojan called "Mac Defender," according to investigations conducted separately by Ed Bott of ZDNet and Jacqui Cheng of Ars Technica. A number of support specialists and Apple Store Geniuses have apparently observed an increase in malware instances on the Mac platform, though others have said that they have not noticed anything amiss. The clincher, however, is an internal memo with a Last Modified date of May 16 from Apple instructing Geniuses not to remove Mac Defender, together with explicit instructions "not to confirm or deny" the presence of the malware. It does appear that Mac Defender is indeed making the rounds.

Ironically, the increasing popularity of MacBook Pro and MacBook Air laptops is no doubt a factor that is attracting hackers to the OS X platform. To be clear, this is by no means an epidemic, though at least one Apple Certified Mac Technician interviewed by Ars Technica says that Mac Defender "appears to have impacted more of the general Mac-using public than all the previous Trojans I've dealt with."
Mac Defender is turning out to be somewhat of an epidemic that neither Apple, nor Mac users seem prepared for. The Mac malware has caught the Apple ecosystem off guard and threatens to shatter the reality distortion field that Apple thrives on.

Apple, and the Apple faithful would like to pretend that Mac malware doesn't exist. But, thanks to some awesome investigative reporting by Ed Bott, Jacqui Cheng, and others, we know that AppleCare technicians are seeing an explosion of malware issues, and that Apple has specifically directed support technicians not to get involved.
.Cheng points out that there is at least tacit acceptance by Apple that the possibility for malware exists because Apple actually sells multiple malware protection products. And, although Apple Store reps are quick to point out the superior security and lack of malware concerns on the Mac, internally Apple mandates the use of Norton malware protection.

A certain Apple loyalist recently called me--and a variety of respected tech writers--out for having the audacity to point out that Mac OS X is not invulnerable and that the potential for Mac malware is steadily rising. Really, I think Gruber was just being lazy that day because his contribution amounts to a single word, and the rest of the article is just a copy and paste of other content like mine.
I get it. Apple products are better. They are magic. They just work. Except when they're not, and they don't. If you look past the smoke and mirrors, you might be disappointed to find out that they're not really that different.

To be fair, I agree that there are fundamental aspects of the OS itself that make Mac OS X more secure, or harder to exploit to some extent. However, I still maintain that the real reason that Macs aren't plagued by more malware is that the platform represents such negligible market share that it's not worth the effort for malware developers.
Ultimately, though, more secure is a long, long way from completely secure or invulnerable. But, many think that Macs are just secure. Period. Mac users who have partaken of that Koolaid and developed some sort of belief that they are impervious to attack by virtue of using a Mac are learning in droves that reality falls short of the myth.

Malware is not the only concern. Many of the biggest threats online are simple phishing attacks and identity theft that are platform agnostic and have nothing to do with the security of the OS. But, the fallacy that Macs are just secure causes users to be naive about security in general. It is time for Mac users to take off the Apple colored glasses and realize that security is a concern.

Don't get me wrong. I like Apple (except for the abomination called iTunes). I love my iPhone and my iPad, and I am seriously considering a MacBook Air. But, I am not naive enough to believe they are secure by default. The false sense of security that comes from believing the Apple hype does users a disservice and leads them to get caught with their proverbial pants down when it comes to the various threats online.