The RSS Newsmaster
The NewsMaster: A New Emergent Socio-Professional Role
In the beginning, it was Yahoo and AltaVista. Then Google came… and the size and the needs grew to such a level of complexity and need for personalization that none of those tools was appropriate anymore.
We are soon to be under enough evolutionary pressure to improve our ability to access specialized news and information in ways and modalities that will make searching with Google or Yahoo appear like a primitive and much-limited approach.
Indeed, the key problem we are confronted with is this: how to navigate, filter, and reduce to humanly manageable dimensions the gigantic flow of information coming at us in increasing amounts.
The entry of the RSS publishing and distribution format into the fray of media and technologies available for communication has only added to the tidal wave of news and information that has gradually been submerging us.
The proliferation of many alternative online independent news sources and the epidemic expansion of the blogger community have created a sharp increase in information channels available to us outside the mainstream, established sources.
Further, the Web keeps growing at a steady pace, and the amount of content being made available on a daily basis is simply too much to be managed by our every day Google and RSS aggregators.
(Nonetheless, the interesting feedback and comments made by Jon Udell and the Scobleizer, my argument does not really center on the stretching of our human abilities to scan thousands of titles while pretending that to be an effective way of searching and finding what is relevant.
If they like to keep on doing that way, I am not against it.)
- The Issues Catalyzing The Change
- The Network Starts To Organize Itself
- RSS Dolby
- What So Great About This?
- What’s The Great Benefit?
- The Network Is Already Doing It!
- So What Is A Newsmaster And What Does He Do?
- Some examples of newsmastering work available out there:
- The NewsMaster Toolkit - An Opportunity To Support Robin Good
- Personal Thoughts
Jakob Nielsen wrote in June of 2003:
“The easier it is to find places with good information, the less time users will spend visiting any individual website. This is one of many conclusions that follow from analyzing how people optimize their behavior in online information systems.”
And you can bet that with the epidemic RSS is having, the exploding amount of content sources, the likely emergence of a multitude of capable newsmasters and innovative technologies supporting them, the number of information seekers, researchers, and normal individuals that used to surf and access tens of sites per day, will, in relative terms, drastically decrease.
If information can come at you in such selected, delicious, ad-free fountains, why would you go surfing or searching the Web?
“Online publishers and e-commerce sites no longer need to see search engines simply as sources of traffic and advertising networks.
For the first time, publishers can use a search engine to filter and deliver relevant content to their readers within their own pages, including customized topic search and syndication.”
Scott Rafer (N/A), Feedster’s CEO said.
We need something of an entirely new order of magnitude to manage all of this information.
Search engines, open directories, and millions of bloggers are not enough.
We need a multi-layered, self-organizing approach that allows the load to be highly distributed and the focus and depth to be guaranteed by the combined result of many highly focused individual efforts.
“The layered mechanism works because at no point is the entire weight of the filtering process concentrated in a single individual or a single resource.
It means that individual agents can work without the need for central control, with the only requirement for a functional system being an open set of connections between the agents.
What RSS does best is that it allows an individual to scan, filter, and pass forward.
That’s all it ever has to do.
The network can and will do the rest.”
The Issues Catalyzing The Change
· Too much information coming in.
· Too many sources to scan.
· General categorization of sources is too limited. By author or publishing house is too limiting. I can’t scan 100 authors a day to see if there is something interesting for me to read or write about.
· Too many new sources coming online daily. Which ones do I pick?
· Difficulty in separating relevant news from press releases, personal rants, and job vacancy announcements
· Too much redundancy - feeds are replicated, the news is duplicated several times, and the same PR stories get republished over and over again.
· Dubious quality of many news sources.
· Selfish. Our search for relevant content and news is an individual activity, in general, with no direct benefit for the community at large
· Non-reusable. Searching for information is mostly carried out as a separate, non-contextual, just-in-time quick personal search.
· Demand for quality, filtered, spam, and ad-free selected content on specialized topics is very high demand. It is, at best, pretty hard to find.
· RSS is the only place where you can start intelligently filtering the ocean of info, but the process and tools presently at hand are limited, and the time required to scan thousands of feeds and titles is non-trivial.
· Google, Yahoo, and all of the other major search engines are not sufficiently powerful to extract and provide quality content on specialized news topics.
They are good at finding online resources and specific content, but when it comes to producing ongoing specialized information channels, these tools do not seem to stand the test of time.
· Bloggers are a valuable and interesting source of independent and alternative information, but it is still very hard to extract valuable content from the thousands of personal rants, superficial notes, and other trivial and research-lacking information items.
RSS newsreaders are still author-based aggregators of information, requiring the news gatherer at bay with hundreds of interesting resources to scan.
News seekers search information by focused topic and not by author. The approach is flawed at its foundations and needs to be rethought in ways that allow for the content that is relevant to me to surface more easily from the whole pool of sources I select.
The Network Starts To Organize Itself
As Stephen Downes mentions in his beautiful post about networks, the network of information needs to organize itself automatically, and we must help it in doing so.
Filtering of the online content happens at different levels, and it needs to be a distributed task.
As Stephen points out, no single resource is capable of handling the oceanic flow of information unaided. And if it does, like Google, it does so often ineffectively.
“Inherent in the structure of the Internet, there are distinct layers of filtering mechanisms, each consisting of a “gather filter forward” mechanism.
At the most basic level the mechanism is fulfilled by a human agent, as in the case of blogs.
In others, it is fulfilled by automatic mechanisms, such in new search tools and online specialized aggregators…
The reason why this system works, while other approaches do not, is that there is no reasonable mechanism which can apply the vast requirements of filtering on a single resource.
If we use metadata, the indexing soon outweighs the content.
If we use search engines, each resource must be subject to extensive analysis to determine context.
What’s important here is that each node of each layer need not worry about the rest, and need not be focused on the goal of the system.
The agent seeks what is available, the way a retinal cell gathers light, and passes on what is relevant, the way a neuron passes on a signal. The filtering occurs not in the individual node, but through the independent actions of the aggregation of nodes.”
RSS Dolby
Bloggers play an important role in this autonomic filtering process, but they are also adding, in the process, lots of extra noise.
It may actually be that to make the filtering process more accurate and precise, the level of noise needs to be increased rather than decreased.
I have recently been noticing how the universe of RSS-based content is also becoming extremely redundant and rich in duplicates and repetitions.
I myself feel guilty as an explorer and early tester of this polluted environment. But as I reason about it with detachment, I realize that it is something indeed inevitable (many others will need to test) and probably beneficial.
The littering and polluting happen as you test how to create new RSS channels and feeds, and when these channels get picked up, exposed, or shared inside a directory or RSS search engine, it becomes very hard to prune and delete them from the RSS universe.
While apparently, this would seem to baffle traditional logic, you have certainly heard about the Dolby sound system, which achieves, through pre-emphasis, a cleaner, hiss-free sound by first adding more noise to the original source and then filtering out with greater ease what superfluous.
So it may indeed be that bloggers add an ergonomic hiss to the information flow, which the upper level of news gatherers and aggregators will actually leverage to filter the actual “valuable content.”
What So Great About This?
The discovery is that we can now start to really organize information in some meaningful, creative, very useful ways.
As smoothly as the words in the previous sentence traverse your mind, you must realize that this is quite an impressive turn we are about to make. Nor is this anything we have seen before happening during this civilization.
No Yahoo, Google, or ODP has or can do what we are about to see.
It is going to be a blossoming of specialized news channels like nothing we have seen before.
The discovery is, in fact, in the unlimited and yet untapped power we now have to search, filter, aggregate, and create focused news/information channels with the only support of our know-how, culture, experience, and a little unknown-free technology called RSS.
What’s The Great Benefit?
The real value is in the fact that the effective, widespread application of this filtering and redistribution process would be a highly valuable social endeavor.
As amazing as it may appear, the emergence of the newsmasters shows that individualistic and self-directed work may actually provide great social benefits.
The creation of dedicated information channels, originated by independent publishers and not by vested commercial interests or mainstream media conglomerates, may create the opportunity for a true renaissance of culture learning and a multiplication of our abilities to manage large amounts of rapidly changing information.
Search specialists and librarians who will craft with time investments appropriate queries to get at the information they were looking for will be generously rewarded with eternal fountains of relevant info for the time to come.
The act of search evolves from a mere set of one-time shots at finding something just in time to the start of a collective refined meta-filtering process that generates better information for everyone.
The more refined filtering process that the newsmaster creates can benefit private enterprises but can also benefit, for an extended time and with great value, the research and learning needs of free individuals not connected to direct commercial interests.
Having a great flow of information that can be tapped, aggregated, and filtered accurately is a benefit for all. Its uses and applications are limitless, and they can not only enrich our rapidly evolving global culture but can also give the opportunity for ethical sustainability to those who have the will and skill to leverage them.
Again:
“What’s important here is that each node of each layer need not worry about the rest and need not be focused on the goal of the system.
The agent seeks what is available, the way a retinal cell gathers light, and passes on what is relevant, the way a neuron passes on a signal.
The filtering occurs not in the individual node, but through the independent actions of the aggregation of nodes.”
The Network Is Already Doing It!
At the most basic level, bloggers represent the first generation of self-organizers of the network. If we exclude Yahoo's initial honorable effort, a similar one (e.g., ODP), there is nothing else that comes close in scope to what the universe of bloggers or noosphere has been able to achieve.
Bloggers have been first to scout new independent sources alternative voices, and who have pointed links to new ways of looking at any issue.
They, more than any other group, have brought, among much useless noise, the true emergence of effective meta-news sources that originate, filter, and aggregate valuable content online.
“Decisions about selection and classification are made on a case by case basis using very coarse, and unregulated, mechanisms.”
So, what appears to the many superficial onlookers as a universe of mindless writings (blogs) is nothing less than the initial phase of a complex and orderly process whereby humanity at large takes control of filtering, gathering, and re-organizing his own know-how and discoveries.
Newsmasters may indeed represent the second layer of filtering that we can now apply to this ocean of content and information. …and Newsmasters, as it appears, have been indeed emerging and quietly working at this nouvelle craft for quite some time now.
With the above, the emergence of a new breed of tools, tuned to provide the specific facilities required by the fascinating job of the Newsmaster, heralds the beginning of a popularization process that will allow, like in the past, for DTP (desktop publishing), those who have the skill, to become the true competitors of any mainstream information source.
Now that we have stripped information from everything that was superfluous to it, the real value and skill is in the true ability to find, pick, select, and bring together intelligently the best information bits available out there.
And due to the unique nature of RSS-based content and of its user-centric character, what Newsmasters create are much like geysers of information, open hoses that generate uninterrupted useful info on a specific topic. Could we say “fountains of knowledge”?
From what I can infer, it is also likely that in the near future, once Newsmasters are an established profession, end users themselves will start to apply on their own an extra level of more sophisticated aggregation and filtering to the highly specialized feeds they select.
So What Is A Newsmaster And What Does He Do?
NewsMaster definition:
The newsmaster is an individual capable of personally crafting RSS-based specialized information channels by utilizing technologies that allow hir to select, aggregate, filter, exclude, and identify quality news, information, content, tools, and resources from the whole universe of content, news, and information available on the Internet.
Newsmastering is the ability of a human being to concert, orchestrate, edit, and refine quality search formulas that tap into the whole Internet content universe and beyond and filter out relevant information through selected keywords, source selection, ranking, heuristics, and many other possible criteria.
The Newsmaster specializes in crafting uniquely powerful search/aggregation/filtering formulas generating continuous RSS feeds on narrowly selected topics by:
a) Resource identification. Selecting and aggregating valuable resources (like a normal aggregator does)
b) Query formulation at different levels and on different content bases.
1) Creating advanced search queries in the blogosphere ( a la Bloglines
2) Creating advanced search queries on the traditional Web content at large
3) Creating advanced search queries on the overall pot of content derived at points a), b), and c) and generating new highly filtered RSS feeds matching specific content and quality criteria.
4) Creating advanced search queries to the deep/invisible Web with dedicated
tools and creating RSS content out of the results obtained.
f) Filtering. Applying exclusion filters based on keywords, keyphrases, date, language, and more.
g) Splicing (union): I want feed C to be the result of merging feeds A and B.
h) Intersecting: Given primary feeds A and B, I want feed C to consist of all items that appear in both primary feeds.
i) Subtracting (difference): I want to remove from feed A all of the items that also appear in feed B. Put the result in feed C.
l) Splitting (subset selection): I want to split feed D into feeds D1 and D2 according to some binary selection criteria on items.
g) Heuristics. Utilization of heuristics allows the system to learn from previous choices the patterns characterizing the most appropriate news items from the rest.
i) Reputation. Possible utilization of real-time collaborative filtering mechanisms allowing the newsmaster to benefit also from the rating and reputation given to selected news/information items by other independent newsmasters.
f) Personal selection. The ability to integrate into the newsmastering workflow the ability to personally select information items or to continually improve and refine the governing search and aggregation formulas by way of personal evaluation.
Some Examples of Newsmastering Work Available Out There:
Here are some living examples of newsmastering work available out there:
1) Stephen Downes
Edu_RSS (N/A)
2) Harry Potter channel
https://web.archive.org/web/20041027084633/http://mywireservice.com/sources/view?feedid=400
3) MasterViews
https://web.archive.org/web/20110301030422/http://masterview.ikonosnewmedia.com/
A timid initial approach by the author to newsmastering with the automatic generation of a news page focusing on presentation, information design, and PowerPoint resources and skills.
4) Infosecdaily
https://web.archive.org/web/20040411074829/http://infosecdaily.net/
Over 11 dozen news feeds and over 100 blogs aggregated to give a picture of the information security space.
Nothing non-commercial met these needs; I don’t think anything else does yet. (by Jose Nazario)
5) Monkey News
http://monkey.org/~jose/news/aggregated.html
A personal world news system aggregating close to 100 news feeds.
Through heuristics, it identifies interesting news items and sorts the results in that order. Got sick of Google News’ fluffy feel? Here is the alternative. (by Jose Nazario)
6) Please suggest more! I know there must be thousands out there!
The NewsMaster Toolkit - An Opportunity To Support Robin Good
In an effort to offer you an elegant way to support my further research and reporting in this direction, I have made available a small report entitled the Newsmaster Toolkit.
It now contains over 40 different selected tools that can be used for experimenting and fully testing the thrill of newsmastering today. With your personal skills and these technologies, you can start building dedicated news channels and focussed mini websites in little time.
Once again, don’t get me wrong. The toolkit is an opportunity I am offering you to support my reporting in this area. If you want to thank, reward, encourage, or simply acknowledge the value of this, please get yourself a Toolkit today.
More information about the NewsMaster Toolkit is here.
Personal Thoughts
Newsmasters and their role are central to my position as an evangelist of intelligent use of new media technologies.
Here is a great opportunity to be taken: How to intelligently use the power of RSS-based content to facilitate more effective learning and communication by initiating a process of self-organization of all of the content available on the Internet.
I would suggest the ones of you that like what I am describing start working and getting skilled at this as soon as possible.
Being there at the beginning and understanding early in the game the principles of this fascinating mechanism can only provide you with a greater ability to manage and craft more effective tools rather than having to follow what the market will make available for everyone.
At any rate, the desired asset will not be the one of only having a better or more evolved technological toolset, but rather the combined personal ability and skill to filter, select, pick, and edit/comment on those news, ideas, information, and resources that are most valuable to your specialized information channel(s).
While technology will certainly greatly contribute to our ability to rapidly select and manage while drawing from a large pool of sources, it will be our personal experience, know-how, and intuition that will guide us in customizing the filtering mechanisms we will create.
So if you are into newsmastering, you need both to learn and refine your research skills applied to this new vast set of information sources now available, as well as to facilitate the development and refinement of technologies that ease the tremendous obelizing task at hand (picking through thousands of RSS-based content feeds and filtering out the pure junk from what may be just what you are looking for).
Originally written by Robin Good for MasterNewMedia and first published on Tuesday, March 2, 2004, as “The RSS NewsMaster.”
Robin Good
